2023-11-02 03:53:15,467 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-11-02 03:53:15,717 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438 2023-11-02 03:53:15,731 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-11-02 03:53:15,731 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/cluster_a48e5c36-4405-e712-b078-c0f3440ea40f, deleteOnExit=true 2023-11-02 03:53:15,732 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-11-02 03:53:15,732 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/test.cache.data in system properties and HBase conf 2023-11-02 03:53:15,732 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/hadoop.tmp.dir in system properties and HBase conf 2023-11-02 03:53:15,733 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/hadoop.log.dir in system properties and HBase conf 2023-11-02 03:53:15,734 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/mapreduce.cluster.local.dir in system properties and HBase conf 2023-11-02 03:53:15,734 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-11-02 03:53:15,734 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-11-02 03:53:15,858 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-11-02 03:53:16,253 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-11-02 03:53:16,259 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-11-02 03:53:16,260 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-11-02 03:53:16,260 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-11-02 03:53:16,261 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-11-02 03:53:16,262 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-11-02 03:53:16,262 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-11-02 03:53:16,262 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-11-02 03:53:16,263 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/dfs.journalnode.edits.dir in system properties and HBase conf 2023-11-02 03:53:16,263 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-11-02 03:53:16,264 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/nfs.dump.dir in system properties and HBase conf 2023-11-02 03:53:16,264 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/java.io.tmpdir in system properties and HBase conf 2023-11-02 03:53:16,265 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/dfs.journalnode.edits.dir in system properties and HBase conf 2023-11-02 03:53:16,265 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-11-02 03:53:16,266 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-11-02 03:53:16,930 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-11-02 03:53:16,935 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-11-02 03:53:17,239 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-11-02 03:53:17,412 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-11-02 03:53:17,435 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-11-02 03:53:17,479 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-11-02 03:53:17,512 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/java.io.tmpdir/Jetty_localhost_45387_hdfs____eybim3/webapp 2023-11-02 03:53:17,661 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45387 2023-11-02 03:53:17,672 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-11-02 03:53:17,673 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-11-02 03:53:18,218 WARN [Listener at localhost/39871] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-11-02 03:53:18,298 WARN [Listener at localhost/39871] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-11-02 03:53:18,322 WARN [Listener at localhost/39871] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-11-02 03:53:18,330 INFO [Listener at localhost/39871] log.Slf4jLog(67): jetty-6.1.26 2023-11-02 03:53:18,337 INFO [Listener at localhost/39871] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/java.io.tmpdir/Jetty_localhost_40473_datanode____.v8vw91/webapp 2023-11-02 03:53:18,441 INFO [Listener at localhost/39871] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40473 2023-11-02 03:53:18,812 WARN [Listener at localhost/43647] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-11-02 03:53:19,376 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb623689d4e437afb: Processing first storage report for DS-a7031bb6-1644-49fe-b5f6-68e248987247 from datanode 3448414c-2f66-47a5-b942-f11f797b9875 2023-11-02 03:53:19,378 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb623689d4e437afb: from storage DS-a7031bb6-1644-49fe-b5f6-68e248987247 node DatanodeRegistration(127.0.0.1:45935, datanodeUuid=3448414c-2f66-47a5-b942-f11f797b9875, infoPort=37231, infoSecurePort=0, ipcPort=43647, storageInfo=lv=-57;cid=testClusterID;nsid=695256238;c=1698897197016), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-11-02 03:53:19,378 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xb623689d4e437afb: Processing first storage report for DS-9ef038fa-be62-4645-b573-759f111eb55d from datanode 3448414c-2f66-47a5-b942-f11f797b9875 2023-11-02 03:53:19,378 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xb623689d4e437afb: from storage DS-9ef038fa-be62-4645-b573-759f111eb55d node DatanodeRegistration(127.0.0.1:45935, datanodeUuid=3448414c-2f66-47a5-b942-f11f797b9875, infoPort=37231, infoSecurePort=0, ipcPort=43647, storageInfo=lv=-57;cid=testClusterID;nsid=695256238;c=1698897197016), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-11-02 03:53:19,390 DEBUG [Listener at localhost/43647] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438 2023-11-02 03:53:19,516 INFO [Listener at localhost/43647] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/cluster_a48e5c36-4405-e712-b078-c0f3440ea40f/zookeeper_0, clientPort=65185, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/cluster_a48e5c36-4405-e712-b078-c0f3440ea40f/zookeeper_0/version-2, dataDirSize=457 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/cluster_a48e5c36-4405-e712-b078-c0f3440ea40f/zookeeper_0/version-2, dataLogSize=457 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, clientPortListenBacklog=-1, serverId=0 2023-11-02 03:53:19,535 INFO [Listener at localhost/43647] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=65185 2023-11-02 03:53:19,543 INFO [Listener at localhost/43647] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-02 03:53:19,546 INFO [Listener at localhost/43647] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-02 03:53:20,230 INFO [Listener at localhost/43647] util.FSUtils(471): Created version file at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb with version=8 2023-11-02 03:53:20,230 INFO [Listener at localhost/43647] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/hbase-staging 2023-11-02 03:53:20,573 INFO [Listener at localhost/43647] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-11-02 03:53:21,015 INFO [Listener at localhost/43647] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-11-02 03:53:21,047 INFO [Listener at localhost/43647] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-02 03:53:21,047 INFO [Listener at localhost/43647] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-11-02 03:53:21,048 INFO [Listener at localhost/43647] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-11-02 03:53:21,048 INFO [Listener at localhost/43647] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-02 03:53:21,048 INFO [Listener at localhost/43647] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-11-02 03:53:21,194 INFO [Listener at localhost/43647] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-11-02 03:53:21,263 DEBUG [Listener at localhost/43647] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-11-02 03:53:21,362 INFO [Listener at localhost/43647] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41275 2023-11-02 03:53:21,372 INFO [Listener at localhost/43647] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-02 03:53:21,374 INFO [Listener at localhost/43647] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-02 03:53:21,398 INFO [Listener at localhost/43647] zookeeper.RecoverableZooKeeper(93): Process identifier=master:41275 connecting to ZooKeeper ensemble=127.0.0.1:65185 2023-11-02 03:53:21,448 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:412750x0, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-11-02 03:53:21,451 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:41275-0x101a4e939270000 connected 2023-11-02 03:53:21,481 DEBUG [Listener at localhost/43647] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-11-02 03:53:21,482 DEBUG [Listener at localhost/43647] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-02 03:53:21,485 DEBUG [Listener at localhost/43647] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-11-02 03:53:21,493 DEBUG [Listener at localhost/43647] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41275 2023-11-02 03:53:21,493 DEBUG [Listener at localhost/43647] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41275 2023-11-02 03:53:21,494 DEBUG [Listener at localhost/43647] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41275 2023-11-02 03:53:21,495 DEBUG [Listener at localhost/43647] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41275 2023-11-02 03:53:21,497 DEBUG [Listener at localhost/43647] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41275 2023-11-02 03:53:21,502 INFO [Listener at localhost/43647] master.HMaster(444): hbase.rootdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb, hbase.cluster.distributed=false 2023-11-02 03:53:21,575 INFO [Listener at localhost/43647] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-11-02 03:53:21,575 INFO [Listener at localhost/43647] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-02 03:53:21,576 INFO [Listener at localhost/43647] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-11-02 03:53:21,576 INFO [Listener at localhost/43647] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-11-02 03:53:21,576 INFO [Listener at localhost/43647] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-11-02 03:53:21,576 INFO [Listener at localhost/43647] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-11-02 03:53:21,581 INFO [Listener at localhost/43647] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-11-02 03:53:21,584 INFO [Listener at localhost/43647] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36627 2023-11-02 03:53:21,586 INFO [Listener at localhost/43647] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-11-02 03:53:21,592 DEBUG [Listener at localhost/43647] mob.MobFileCache(121): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-11-02 03:53:21,593 INFO [Listener at localhost/43647] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-02 03:53:21,596 INFO [Listener at localhost/43647] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-02 03:53:21,598 INFO [Listener at localhost/43647] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36627 connecting to ZooKeeper ensemble=127.0.0.1:65185 2023-11-02 03:53:21,614 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:366270x0, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-11-02 03:53:21,615 DEBUG [Listener at localhost/43647] zookeeper.ZKUtil(164): regionserver:366270x0, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-11-02 03:53:21,621 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:36627-0x101a4e939270001 connected 2023-11-02 03:53:21,623 DEBUG [Listener at localhost/43647] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-02 03:53:21,624 DEBUG [Listener at localhost/43647] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-11-02 03:53:21,644 DEBUG [Listener at localhost/43647] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36627 2023-11-02 03:53:21,649 DEBUG [Listener at localhost/43647] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36627 2023-11-02 03:53:21,658 DEBUG [Listener at localhost/43647] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36627 2023-11-02 03:53:21,671 DEBUG [Listener at localhost/43647] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36627 2023-11-02 03:53:21,675 DEBUG [Listener at localhost/43647] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36627 2023-11-02 03:53:21,677 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,41275,1698897200405 2023-11-02 03:53:21,688 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-11-02 03:53:21,691 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,41275,1698897200405 2023-11-02 03:53:21,711 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-11-02 03:53:21,711 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-11-02 03:53:21,711 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-02 03:53:21,712 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-11-02 03:53:21,713 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-11-02 03:53:21,714 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,41275,1698897200405 from backup master directory 2023-11-02 03:53:21,717 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,41275,1698897200405 2023-11-02 03:53:21,717 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-11-02 03:53:21,718 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-11-02 03:53:21,718 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,41275,1698897200405 2023-11-02 03:53:21,721 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-11-02 03:53:21,723 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-11-02 03:53:21,803 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/hbase.id with ID: a4fba843-ea50-42c2-a0cd-bb7769697865 2023-11-02 03:53:21,844 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-02 03:53:21,862 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-02 03:53:21,902 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x73f47c81 to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:53:21,935 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@199f05a4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:53:21,961 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-11-02 03:53:21,963 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-11-02 03:53:21,980 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-11-02 03:53:21,980 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-11-02 03:53:21,982 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-11-02 03:53:21,985 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-11-02 03:53:21,987 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-11-02 03:53:22,020 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/MasterData/data/master/store-tmp 2023-11-02 03:53:22,047 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:53:22,047 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-11-02 03:53:22,047 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-02 03:53:22,048 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-02 03:53:22,048 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-11-02 03:53:22,048 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-02 03:53:22,048 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-02 03:53:22,048 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-11-02 03:53:22,050 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/MasterData/WALs/jenkins-hbase4.apache.org,41275,1698897200405 2023-11-02 03:53:22,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41275%2C1698897200405, suffix=, logDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/MasterData/WALs/jenkins-hbase4.apache.org,41275,1698897200405, archiveDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/MasterData/oldWALs, maxLogs=10 2023-11-02 03:53:22,158 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45935,DS-a7031bb6-1644-49fe-b5f6-68e248987247,DISK] 2023-11-02 03:53:22,168 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-11-02 03:53:22,247 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/MasterData/WALs/jenkins-hbase4.apache.org,41275,1698897200405/jenkins-hbase4.apache.org%2C41275%2C1698897200405.1698897202091 2023-11-02 03:53:22,248 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45935,DS-a7031bb6-1644-49fe-b5f6-68e248987247,DISK]] 2023-11-02 03:53:22,248 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-11-02 03:53:22,249 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:53:22,253 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-11-02 03:53:22,255 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-11-02 03:53:22,321 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-11-02 03:53:22,328 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-11-02 03:53:22,353 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-11-02 03:53:22,365 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:53:22,372 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-11-02 03:53:22,374 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-11-02 03:53:22,392 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-11-02 03:53:22,397 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:53:22,398 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67775411, jitterRate=0.009932324290275574}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-02 03:53:22,398 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-11-02 03:53:22,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-11-02 03:53:22,417 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-11-02 03:53:22,418 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(561): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-11-02 03:53:22,420 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-11-02 03:53:22,422 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(581): Recovered RegionProcedureStore lease in 1 msec 2023-11-02 03:53:22,456 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(595): Loaded RegionProcedureStore in 33 msec 2023-11-02 03:53:22,456 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-11-02 03:53:22,483 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-11-02 03:53:22,490 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-11-02 03:53:22,515 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-11-02 03:53:22,519 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-11-02 03:53:22,521 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-11-02 03:53:22,527 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-11-02 03:53:22,532 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-11-02 03:53:22,536 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-02 03:53:22,538 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-11-02 03:53:22,538 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-11-02 03:53:22,551 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-11-02 03:53:22,556 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-11-02 03:53:22,557 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-11-02 03:53:22,557 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-02 03:53:22,559 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,41275,1698897200405, sessionid=0x101a4e939270000, setting cluster-up flag (Was=false) 2023-11-02 03:53:22,576 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-02 03:53:22,583 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-11-02 03:53:22,585 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,41275,1698897200405 2023-11-02 03:53:22,592 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-02 03:53:22,597 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-11-02 03:53:22,599 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,41275,1698897200405 2023-11-02 03:53:22,601 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(304): Couldn't delete working snapshot directory: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.hbase-snapshot/.tmp 2023-11-02 03:53:22,682 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(951): ClusterId : a4fba843-ea50-42c2-a0cd-bb7769697865 2023-11-02 03:53:22,690 DEBUG [RS:0;jenkins-hbase4:36627] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-11-02 03:53:22,701 DEBUG [RS:0;jenkins-hbase4:36627] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-11-02 03:53:22,701 DEBUG [RS:0;jenkins-hbase4:36627] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-11-02 03:53:22,708 DEBUG [RS:0;jenkins-hbase4:36627] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-11-02 03:53:22,710 DEBUG [RS:0;jenkins-hbase4:36627] zookeeper.ReadOnlyZKClient(139): Connect 0x320de6d4 to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:53:22,717 DEBUG [RS:0;jenkins-hbase4:36627] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3cca1ee5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:53:22,717 DEBUG [RS:0;jenkins-hbase4:36627] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@38f58da4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-11-02 03:53:22,748 DEBUG [RS:0;jenkins-hbase4:36627] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:36627 2023-11-02 03:53:22,752 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-11-02 03:53:22,753 INFO [RS:0;jenkins-hbase4:36627] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-11-02 03:53:22,753 INFO [RS:0;jenkins-hbase4:36627] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-11-02 03:53:22,753 DEBUG [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(1022): About to register with Master. 2023-11-02 03:53:22,757 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,41275,1698897200405 with isa=jenkins-hbase4.apache.org/172.31.14.131:36627, startcode=1698897201574 2023-11-02 03:53:22,766 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-11-02 03:53:22,766 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-11-02 03:53:22,766 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-11-02 03:53:22,766 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-11-02 03:53:22,766 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-11-02 03:53:22,766 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-02 03:53:22,766 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-11-02 03:53:22,766 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-02 03:53:22,774 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1698897232774 2023-11-02 03:53:22,775 DEBUG [RS:0;jenkins-hbase4:36627] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-11-02 03:53:22,776 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-11-02 03:53:22,778 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-11-02 03:53:22,778 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-11-02 03:53:22,784 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-11-02 03:53:22,789 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-11-02 03:53:22,796 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-11-02 03:53:22,797 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-11-02 03:53:22,799 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-11-02 03:53:22,799 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-11-02 03:53:22,799 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-11-02 03:53:22,801 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-11-02 03:53:22,803 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-11-02 03:53:22,803 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-11-02 03:53:22,810 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-11-02 03:53:22,810 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-11-02 03:53:22,813 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1698897202813,5,FailOnTimeoutGroup] 2023-11-02 03:53:22,814 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1698897202814,5,FailOnTimeoutGroup] 2023-11-02 03:53:22,815 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-11-02 03:53:22,816 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-11-02 03:53:22,818 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-11-02 03:53:22,819 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-11-02 03:53:22,821 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-11-02 03:53:22,822 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-11-02 03:53:22,823 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb 2023-11-02 03:53:22,837 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:53:22,840 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-11-02 03:53:22,841 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60041, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-11-02 03:53:22,843 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/info 2023-11-02 03:53:22,844 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-11-02 03:53:22,845 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:53:22,846 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-11-02 03:53:22,848 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/rep_barrier 2023-11-02 03:53:22,849 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-11-02 03:53:22,850 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:53:22,851 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-11-02 03:53:22,853 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41275] master.ServerManager(388): Registering regionserver=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:22,854 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/table 2023-11-02 03:53:22,855 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-11-02 03:53:22,856 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:53:22,858 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740 2023-11-02 03:53:22,859 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740 2023-11-02 03:53:22,863 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-11-02 03:53:22,866 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-11-02 03:53:22,870 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:53:22,872 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65515270, jitterRate=-0.023746401071548462}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-11-02 03:53:22,872 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-11-02 03:53:22,872 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-11-02 03:53:22,872 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-11-02 03:53:22,872 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-11-02 03:53:22,872 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-11-02 03:53:22,872 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-11-02 03:53:22,873 DEBUG [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb 2023-11-02 03:53:22,873 DEBUG [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:39871 2023-11-02 03:53:22,873 DEBUG [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-11-02 03:53:22,873 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-11-02 03:53:22,874 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-11-02 03:53:22,880 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-11-02 03:53:22,880 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-11-02 03:53:22,880 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-11-02 03:53:22,881 DEBUG [RS:0;jenkins-hbase4:36627] zookeeper.ZKUtil(162): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:22,881 WARN [RS:0;jenkins-hbase4:36627] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-11-02 03:53:22,884 INFO [RS:0;jenkins-hbase4:36627] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-11-02 03:53:22,885 DEBUG [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(1948): logDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/WALs/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:22,886 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36627,1698897201574] 2023-11-02 03:53:22,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-11-02 03:53:22,896 DEBUG [RS:0;jenkins-hbase4:36627] zookeeper.ZKUtil(162): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:22,904 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-11-02 03:53:22,906 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-11-02 03:53:22,910 DEBUG [RS:0;jenkins-hbase4:36627] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-11-02 03:53:22,919 INFO [RS:0;jenkins-hbase4:36627] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-11-02 03:53:22,938 INFO [RS:0;jenkins-hbase4:36627] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-11-02 03:53:22,942 INFO [RS:0;jenkins-hbase4:36627] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-02 03:53:22,942 INFO [RS:0;jenkins-hbase4:36627] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-02 03:53:22,943 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-11-02 03:53:22,950 INFO [RS:0;jenkins-hbase4:36627] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-11-02 03:53:22,951 DEBUG [RS:0;jenkins-hbase4:36627] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-02 03:53:22,951 DEBUG [RS:0;jenkins-hbase4:36627] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-02 03:53:22,951 DEBUG [RS:0;jenkins-hbase4:36627] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-02 03:53:22,951 DEBUG [RS:0;jenkins-hbase4:36627] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-02 03:53:22,951 DEBUG [RS:0;jenkins-hbase4:36627] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-02 03:53:22,951 DEBUG [RS:0;jenkins-hbase4:36627] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-11-02 03:53:22,951 DEBUG [RS:0;jenkins-hbase4:36627] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-02 03:53:22,951 DEBUG [RS:0;jenkins-hbase4:36627] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-02 03:53:22,952 DEBUG [RS:0;jenkins-hbase4:36627] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-02 03:53:22,952 DEBUG [RS:0;jenkins-hbase4:36627] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-11-02 03:53:22,952 INFO [RS:0;jenkins-hbase4:36627] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-11-02 03:53:22,953 INFO [RS:0;jenkins-hbase4:36627] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-11-02 03:53:22,953 INFO [RS:0;jenkins-hbase4:36627] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-11-02 03:53:22,970 INFO [RS:0;jenkins-hbase4:36627] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-11-02 03:53:22,972 INFO [RS:0;jenkins-hbase4:36627] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36627,1698897201574-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-11-02 03:53:22,989 INFO [RS:0;jenkins-hbase4:36627] regionserver.Replication(203): jenkins-hbase4.apache.org,36627,1698897201574 started 2023-11-02 03:53:22,990 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,36627,1698897201574, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36627, sessionid=0x101a4e939270001 2023-11-02 03:53:22,990 DEBUG [RS:0;jenkins-hbase4:36627] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-11-02 03:53:22,990 DEBUG [RS:0;jenkins-hbase4:36627] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:22,990 DEBUG [RS:0;jenkins-hbase4:36627] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36627,1698897201574' 2023-11-02 03:53:22,990 DEBUG [RS:0;jenkins-hbase4:36627] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:22,991 DEBUG [RS:0;jenkins-hbase4:36627] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:22,992 DEBUG [RS:0;jenkins-hbase4:36627] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-11-02 03:53:22,992 DEBUG [RS:0;jenkins-hbase4:36627] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-11-02 03:53:22,992 DEBUG [RS:0;jenkins-hbase4:36627] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:22,992 DEBUG [RS:0;jenkins-hbase4:36627] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36627,1698897201574' 2023-11-02 03:53:22,992 DEBUG [RS:0;jenkins-hbase4:36627] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-11-02 03:53:22,993 DEBUG [RS:0;jenkins-hbase4:36627] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-11-02 03:53:22,994 DEBUG [RS:0;jenkins-hbase4:36627] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-11-02 03:53:22,994 INFO [RS:0;jenkins-hbase4:36627] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-11-02 03:53:22,994 INFO [RS:0;jenkins-hbase4:36627] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-11-02 03:53:23,058 DEBUG [jenkins-hbase4:41275] assignment.AssignmentManager(2186): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-11-02 03:53:23,061 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36627,1698897201574, state=OPENING 2023-11-02 03:53:23,070 DEBUG [PEWorker-4] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-11-02 03:53:23,072 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-02 03:53:23,073 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-11-02 03:53:23,077 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:53:23,104 INFO [RS:0;jenkins-hbase4:36627] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36627%2C1698897201574, suffix=, logDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/WALs/jenkins-hbase4.apache.org,36627,1698897201574, archiveDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/oldWALs, maxLogs=32 2023-11-02 03:53:23,125 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45935,DS-a7031bb6-1644-49fe-b5f6-68e248987247,DISK] 2023-11-02 03:53:23,129 INFO [RS:0;jenkins-hbase4:36627] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/WALs/jenkins-hbase4.apache.org,36627,1698897201574/jenkins-hbase4.apache.org%2C36627%2C1698897201574.1698897203108 2023-11-02 03:53:23,129 DEBUG [RS:0;jenkins-hbase4:36627] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45935,DS-a7031bb6-1644-49fe-b5f6-68e248987247,DISK]] 2023-11-02 03:53:23,266 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:23,268 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-11-02 03:53:23,272 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50710, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-11-02 03:53:23,283 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-11-02 03:53:23,283 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-11-02 03:53:23,287 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36627%2C1698897201574.meta, suffix=.meta, logDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/WALs/jenkins-hbase4.apache.org,36627,1698897201574, archiveDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/oldWALs, maxLogs=32 2023-11-02 03:53:23,305 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:45935,DS-a7031bb6-1644-49fe-b5f6-68e248987247,DISK] 2023-11-02 03:53:23,310 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/WALs/jenkins-hbase4.apache.org,36627,1698897201574/jenkins-hbase4.apache.org%2C36627%2C1698897201574.meta.1698897203289.meta 2023-11-02 03:53:23,310 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:45935,DS-a7031bb6-1644-49fe-b5f6-68e248987247,DISK]] 2023-11-02 03:53:23,310 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-11-02 03:53:23,312 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-11-02 03:53:23,329 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-11-02 03:53:23,335 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-11-02 03:53:23,342 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-11-02 03:53:23,342 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:53:23,342 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-11-02 03:53:23,342 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-11-02 03:53:23,345 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-11-02 03:53:23,347 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/info 2023-11-02 03:53:23,347 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/info 2023-11-02 03:53:23,347 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-11-02 03:53:23,348 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:53:23,348 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-11-02 03:53:23,350 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/rep_barrier 2023-11-02 03:53:23,350 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/rep_barrier 2023-11-02 03:53:23,350 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-11-02 03:53:23,351 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:53:23,352 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-11-02 03:53:23,354 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/table 2023-11-02 03:53:23,354 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/table 2023-11-02 03:53:23,355 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-11-02 03:53:23,356 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:53:23,357 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740 2023-11-02 03:53:23,360 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740 2023-11-02 03:53:23,363 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-11-02 03:53:23,366 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-11-02 03:53:23,367 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64170540, jitterRate=-0.04378443956375122}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-11-02 03:53:23,368 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-11-02 03:53:23,377 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1698897203258 2023-11-02 03:53:23,394 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-11-02 03:53:23,395 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-11-02 03:53:23,395 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,36627,1698897201574, state=OPEN 2023-11-02 03:53:23,401 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-11-02 03:53:23,401 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-11-02 03:53:23,406 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=3, resume processing ppid=2 2023-11-02 03:53:23,406 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,36627,1698897201574 in 324 msec 2023-11-02 03:53:23,412 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=2, resume processing ppid=1 2023-11-02 03:53:23,412 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 517 msec 2023-11-02 03:53:23,418 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 759 msec 2023-11-02 03:53:23,419 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1698897203418, completionTime=-1 2023-11-02 03:53:23,419 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(808): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-11-02 03:53:23,419 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1527): Joining cluster... 2023-11-02 03:53:23,485 DEBUG [hconnection-0x1644e549-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-02 03:53:23,488 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50714, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-02 03:53:23,509 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1539): Number of RegionServers=1 2023-11-02 03:53:23,509 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1698897263509 2023-11-02 03:53:23,510 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1698897323510 2023-11-02 03:53:23,510 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1546): Joined the cluster in 90 msec 2023-11-02 03:53:23,532 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41275,1698897200405-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-11-02 03:53:23,532 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41275,1698897200405-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-11-02 03:53:23,532 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41275,1698897200405-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-11-02 03:53:23,534 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:41275, period=300000, unit=MILLISECONDS is enabled. 2023-11-02 03:53:23,534 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-11-02 03:53:23,540 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-11-02 03:53:23,548 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-11-02 03:53:23,549 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-11-02 03:53:23,558 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-11-02 03:53:23,560 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-11-02 03:53:23,563 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-02 03:53:23,585 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/hbase/namespace/e9d73087146f1976779ec867087e57e1 2023-11-02 03:53:23,587 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/hbase/namespace/e9d73087146f1976779ec867087e57e1 empty. 2023-11-02 03:53:23,588 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/hbase/namespace/e9d73087146f1976779ec867087e57e1 2023-11-02 03:53:23,588 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-11-02 03:53:24,012 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-11-02 03:53:24,014 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => e9d73087146f1976779ec867087e57e1, NAME => 'hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp 2023-11-02 03:53:24,042 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:53:24,042 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing e9d73087146f1976779ec867087e57e1, disabling compactions & flushes 2023-11-02 03:53:24,042 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1. 2023-11-02 03:53:24,042 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1. 2023-11-02 03:53:24,043 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1. after waiting 0 ms 2023-11-02 03:53:24,043 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1. 2023-11-02 03:53:24,043 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1. 2023-11-02 03:53:24,043 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for e9d73087146f1976779ec867087e57e1: 2023-11-02 03:53:24,050 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-11-02 03:53:24,066 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1698897204054"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897204054"}]},"ts":"1698897204054"} 2023-11-02 03:53:24,099 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-02 03:53:24,101 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-02 03:53:24,106 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897204102"}]},"ts":"1698897204102"} 2023-11-02 03:53:24,110 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-11-02 03:53:24,118 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=e9d73087146f1976779ec867087e57e1, ASSIGN}] 2023-11-02 03:53:24,126 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=e9d73087146f1976779ec867087e57e1, ASSIGN 2023-11-02 03:53:24,130 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=e9d73087146f1976779ec867087e57e1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=false 2023-11-02 03:53:24,281 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=e9d73087146f1976779ec867087e57e1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:24,282 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1698897204281"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897204281"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897204281"}]},"ts":"1698897204281"} 2023-11-02 03:53:24,289 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure e9d73087146f1976779ec867087e57e1, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:53:24,445 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:24,451 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1. 2023-11-02 03:53:24,452 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e9d73087146f1976779ec867087e57e1, NAME => 'hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1.', STARTKEY => '', ENDKEY => ''} 2023-11-02 03:53:24,453 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace e9d73087146f1976779ec867087e57e1 2023-11-02 03:53:24,453 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:53:24,453 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e9d73087146f1976779ec867087e57e1 2023-11-02 03:53:24,453 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e9d73087146f1976779ec867087e57e1 2023-11-02 03:53:24,456 INFO [StoreOpener-e9d73087146f1976779ec867087e57e1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region e9d73087146f1976779ec867087e57e1 2023-11-02 03:53:24,458 DEBUG [StoreOpener-e9d73087146f1976779ec867087e57e1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/namespace/e9d73087146f1976779ec867087e57e1/info 2023-11-02 03:53:24,458 DEBUG [StoreOpener-e9d73087146f1976779ec867087e57e1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/namespace/e9d73087146f1976779ec867087e57e1/info 2023-11-02 03:53:24,459 INFO [StoreOpener-e9d73087146f1976779ec867087e57e1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e9d73087146f1976779ec867087e57e1 columnFamilyName info 2023-11-02 03:53:24,460 INFO [StoreOpener-e9d73087146f1976779ec867087e57e1-1] regionserver.HStore(310): Store=e9d73087146f1976779ec867087e57e1/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:53:24,462 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/namespace/e9d73087146f1976779ec867087e57e1 2023-11-02 03:53:24,463 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/namespace/e9d73087146f1976779ec867087e57e1 2023-11-02 03:53:24,469 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e9d73087146f1976779ec867087e57e1 2023-11-02 03:53:24,473 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/namespace/e9d73087146f1976779ec867087e57e1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:53:24,474 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e9d73087146f1976779ec867087e57e1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67660785, jitterRate=0.008224263787269592}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-02 03:53:24,474 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e9d73087146f1976779ec867087e57e1: 2023-11-02 03:53:24,477 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1., pid=6, masterSystemTime=1698897204445 2023-11-02 03:53:24,482 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1. 2023-11-02 03:53:24,482 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1. 2023-11-02 03:53:24,483 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=e9d73087146f1976779ec867087e57e1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:24,484 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1698897204482"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897204482"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897204482"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897204482"}]},"ts":"1698897204482"} 2023-11-02 03:53:24,493 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=6, resume processing ppid=5 2023-11-02 03:53:24,493 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure e9d73087146f1976779ec867087e57e1, server=jenkins-hbase4.apache.org,36627,1698897201574 in 200 msec 2023-11-02 03:53:24,497 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=5, resume processing ppid=4 2023-11-02 03:53:24,497 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=e9d73087146f1976779ec867087e57e1, ASSIGN in 375 msec 2023-11-02 03:53:24,499 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-02 03:53:24,499 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897204499"}]},"ts":"1698897204499"} 2023-11-02 03:53:24,502 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-11-02 03:53:24,507 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-11-02 03:53:24,511 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 957 msec 2023-11-02 03:53:24,561 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-11-02 03:53:24,562 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-11-02 03:53:24,563 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-02 03:53:24,609 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-11-02 03:53:24,629 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-11-02 03:53:24,635 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 36 msec 2023-11-02 03:53:24,643 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1028): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-11-02 03:53:24,657 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-11-02 03:53:24,662 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 18 msec 2023-11-02 03:53:24,669 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-11-02 03:53:24,672 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-11-02 03:53:24,673 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.954sec 2023-11-02 03:53:24,674 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-11-02 03:53:24,676 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-11-02 03:53:24,676 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-11-02 03:53:24,677 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41275,1698897200405-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-11-02 03:53:24,677 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41275,1698897200405-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-11-02 03:53:24,688 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-11-02 03:53:24,691 DEBUG [Listener at localhost/43647] zookeeper.ReadOnlyZKClient(139): Connect 0x0e51ea3b to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:53:24,695 DEBUG [Listener at localhost/43647] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7f9c4d78, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:53:24,708 DEBUG [hconnection-0x13db0c6a-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-02 03:53:24,722 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50718, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-02 03:53:24,733 INFO [Listener at localhost/43647] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,41275,1698897200405 2023-11-02 03:53:24,735 DEBUG [Listener at localhost/43647] zookeeper.ReadOnlyZKClient(139): Connect 0x75a9267a to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:53:24,740 DEBUG [Listener at localhost/43647] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@247479d3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:53:24,740 INFO [Listener at localhost/43647] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-02 03:53:24,790 INFO [Listener at localhost/43647] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=214, OpenFileDescriptor=505, MaxFileDescriptor=60000, SystemLoadAverage=360, ProcessCount=170, AvailableMemoryMB=11664 2023-11-02 03:53:24,801 DEBUG [Listener at localhost/43647] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-02 03:53:24,805 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52022, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-02 03:53:24,820 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-02 03:53:24,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:24,824 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-11-02 03:53:24,827 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-02 03:53:24,829 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-11-02 03:53:24,831 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4 2023-11-02 03:53:24,832 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4 empty. 2023-11-02 03:53:24,834 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4 2023-11-02 03:53:24,834 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-11-02 03:53:24,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-11-02 03:53:24,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-11-02 03:53:25,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-11-02 03:53:25,252 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-11-02 03:53:25,254 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 8f6ce6f475709f0e3737fe28a4b8cba4, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp 2023-11-02 03:53:25,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-11-02 03:53:25,668 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:53:25,669 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1604): Closing 8f6ce6f475709f0e3737fe28a4b8cba4, disabling compactions & flushes 2023-11-02 03:53:25,669 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:25,669 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:25,669 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. after waiting 0 ms 2023-11-02 03:53:25,669 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:25,669 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:25,669 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1558): Region close journal for 8f6ce6f475709f0e3737fe28a4b8cba4: 2023-11-02 03:53:25,677 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-11-02 03:53:25,679 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1698897205679"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897205679"}]},"ts":"1698897205679"} 2023-11-02 03:53:25,683 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-02 03:53:25,686 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-02 03:53:25,686 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897205686"}]},"ts":"1698897205686"} 2023-11-02 03:53:25,689 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-11-02 03:53:25,694 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=8f6ce6f475709f0e3737fe28a4b8cba4, ASSIGN}] 2023-11-02 03:53:25,697 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=8f6ce6f475709f0e3737fe28a4b8cba4, ASSIGN 2023-11-02 03:53:25,701 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=8f6ce6f475709f0e3737fe28a4b8cba4, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=false 2023-11-02 03:53:25,852 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=8f6ce6f475709f0e3737fe28a4b8cba4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:25,853 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1698897205852"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897205852"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897205852"}]},"ts":"1698897205852"} 2023-11-02 03:53:25,856 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 8f6ce6f475709f0e3737fe28a4b8cba4, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:53:25,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-11-02 03:53:26,009 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:26,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:26,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8f6ce6f475709f0e3737fe28a4b8cba4, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4.', STARTKEY => '', ENDKEY => ''} 2023-11-02 03:53:26,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 8f6ce6f475709f0e3737fe28a4b8cba4 2023-11-02 03:53:26,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:53:26,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8f6ce6f475709f0e3737fe28a4b8cba4 2023-11-02 03:53:26,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8f6ce6f475709f0e3737fe28a4b8cba4 2023-11-02 03:53:26,019 INFO [StoreOpener-8f6ce6f475709f0e3737fe28a4b8cba4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8f6ce6f475709f0e3737fe28a4b8cba4 2023-11-02 03:53:26,021 DEBUG [StoreOpener-8f6ce6f475709f0e3737fe28a4b8cba4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1 2023-11-02 03:53:26,021 DEBUG [StoreOpener-8f6ce6f475709f0e3737fe28a4b8cba4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1 2023-11-02 03:53:26,024 INFO [StoreOpener-8f6ce6f475709f0e3737fe28a4b8cba4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8f6ce6f475709f0e3737fe28a4b8cba4 columnFamilyName family1 2023-11-02 03:53:26,027 INFO [StoreOpener-8f6ce6f475709f0e3737fe28a4b8cba4-1] regionserver.HStore(310): Store=8f6ce6f475709f0e3737fe28a4b8cba4/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:53:26,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4 2023-11-02 03:53:26,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4 2023-11-02 03:53:26,036 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8f6ce6f475709f0e3737fe28a4b8cba4 2023-11-02 03:53:26,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:53:26,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8f6ce6f475709f0e3737fe28a4b8cba4; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61816249, jitterRate=-0.07886610925197601}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-02 03:53:26,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8f6ce6f475709f0e3737fe28a4b8cba4: 2023-11-02 03:53:26,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4., pid=11, masterSystemTime=1698897206009 2023-11-02 03:53:26,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:26,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:26,046 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=8f6ce6f475709f0e3737fe28a4b8cba4, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:26,047 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1698897206046"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897206046"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897206046"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897206046"}]},"ts":"1698897206046"} 2023-11-02 03:53:26,053 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=11, resume processing ppid=10 2023-11-02 03:53:26,053 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 8f6ce6f475709f0e3737fe28a4b8cba4, server=jenkins-hbase4.apache.org,36627,1698897201574 in 194 msec 2023-11-02 03:53:26,056 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=10, resume processing ppid=9 2023-11-02 03:53:26,056 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=8f6ce6f475709f0e3737fe28a4b8cba4, ASSIGN in 359 msec 2023-11-02 03:53:26,057 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-02 03:53:26,058 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897206057"}]},"ts":"1698897206057"} 2023-11-02 03:53:26,060 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-11-02 03:53:26,063 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-11-02 03:53:26,066 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.2430 sec 2023-11-02 03:53:26,953 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=9 2023-11-02 03:53:26,954 INFO [Listener at localhost/43647] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-11-02 03:53:26,971 DEBUG [hconnection-0x637d7e04-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-02 03:53:26,974 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41216, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-02 03:53:27,119 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:27,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,126 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-02 03:53:27,126 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:27,127 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-02 03:53:27,127 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-02 03:53:27,127 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,128 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,130 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:27,130 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:27,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:27,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:27,130 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:27,130 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:27,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,131 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:27,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,132 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,134 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,134 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-02 03:53:27,134 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:27,134 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-02 03:53:27,135 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:27,135 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:27,135 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:27,136 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. started... 2023-11-02 03:53:27,136 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8f6ce6f475709f0e3737fe28a4b8cba4 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-02 03:53:27,678 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113073c5250cf4124493ac94e5467bcac57d to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113073c5250cf4124493ac94e5467bcac57d 2023-11-02 03:53:27,693 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/4f5bf6d28d6846c79d59ecfa56cf78fb 2023-11-02 03:53:27,703 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/4f5bf6d28d6846c79d59ecfa56cf78fb as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/4f5bf6d28d6846c79d59ecfa56cf78fb 2023-11-02 03:53:27,712 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/4f5bf6d28d6846c79d59ecfa56cf78fb, entries=1000, sequenceid=5, filesize=175.8 K 2023-11-02 03:53:27,715 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 8f6ce6f475709f0e3737fe28a4b8cba4 in 579ms, sequenceid=5, compaction requested=false 2023-11-02 03:53:27,716 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-02 03:53:27,719 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8f6ce6f475709f0e3737fe28a4b8cba4: 2023-11-02 03:53:27,719 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:27,719 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:27,719 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:27,719 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:27,719 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-02 03:53:27,720 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-02 03:53:27,723 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:27,723 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:27,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:27,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:27,724 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,724 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:27,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:27,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:27,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:27,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:27,726 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-02 03:53:27,726 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-02 03:53:27,726 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@10981bf6[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:27,726 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,729 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,729 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-02 03:53:27,729 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-02 03:53:27,729 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:27,729 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:27,730 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:27,732 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-02 03:53:27,732 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:27,732 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:27,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:27,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:27,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:27,732 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-02 03:53:27,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:27,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:27,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:27,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:27,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:27,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:27,737 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-02 03:53:27,737 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:27,737 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-02 03:53:27,737 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:27,738 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,738 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:27,741 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,741 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,741 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:27,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:27,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:27,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:27,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:27,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:27,742 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:27,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:27,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:27,744 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:27,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:27,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:27,753 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:27,753 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,753 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,753 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:27,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:27,754 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:27,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:27,754 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,754 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,754 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,756 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:27,756 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:27,756 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-02 03:53:27,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:27,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:27,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:27,759 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:27,759 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:27,860 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:27,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:27,875 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:27,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:27,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,878 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-02 03:53:27,878 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:27,879 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-02 03:53:27,879 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-02 03:53:27,880 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,880 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,887 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:27,887 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:27,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:27,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:27,888 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:27,888 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:27,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:27,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,889 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,889 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-02 03:53:27,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:27,889 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-02 03:53:27,890 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:27,890 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-02 03:53:27,890 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:27,890 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:27,890 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:27,890 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. started... 2023-11-02 03:53:27,890 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8f6ce6f475709f0e3737fe28a4b8cba4 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-02 03:53:28,317 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128da3232e95eae424db2b22dc9ae696f07 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128da3232e95eae424db2b22dc9ae696f07 2023-11-02 03:53:28,731 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/3fb8ad9e6b4545d59274b9ab6a39eb8d 2023-11-02 03:53:28,743 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/3fb8ad9e6b4545d59274b9ab6a39eb8d as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/3fb8ad9e6b4545d59274b9ab6a39eb8d 2023-11-02 03:53:28,752 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/3fb8ad9e6b4545d59274b9ab6a39eb8d, entries=1, sequenceid=9, filesize=5.0 K 2023-11-02 03:53:28,753 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 8f6ce6f475709f0e3737fe28a4b8cba4 in 863ms, sequenceid=9, compaction requested=false 2023-11-02 03:53:28,753 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8f6ce6f475709f0e3737fe28a4b8cba4: 2023-11-02 03:53:28,754 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:28,754 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:28,754 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:28,754 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:28,754 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-02 03:53:28,754 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-02 03:53:28,757 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:28,757 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:28,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:28,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:28,757 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,757 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:28,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:28,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:28,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:28,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:28,760 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-02 03:53:28,760 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41abbb8f[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:28,760 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-02 03:53:28,761 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,762 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,762 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-02 03:53:28,762 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-02 03:53:28,762 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:28,763 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:28,763 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:28,765 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:28,765 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-02 03:53:28,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:28,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:28,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:28,765 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:28,765 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-02 03:53:28,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:28,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:28,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:28,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:28,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:28,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:28,769 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-02 03:53:28,769 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-02 03:53:28,770 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:28,770 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:28,770 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,770 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:28,773 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,773 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,773 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:28,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:28,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:28,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:28,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:28,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:28,774 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:28,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:28,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:28,779 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:28,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:28,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:28,784 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:28,784 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,784 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:28,784 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:28,784 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:28,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:28,784 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,784 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,785 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,785 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:28,785 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:28,785 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-02 03:53:28,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:28,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:28,785 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-02 03:53:28,785 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:28,786 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:28,786 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:28,864 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-02 03:53:28,886 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:28,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:28,898 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:28,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:28,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,901 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-02 03:53:28,901 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:28,901 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-02 03:53:28,902 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-02 03:53:28,902 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,902 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,904 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:28,904 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:28,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:28,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:28,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,906 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:28,906 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:28,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:28,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,908 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,908 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-02 03:53:28,908 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:28,908 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-02 03:53:28,908 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:28,909 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-02 03:53:28,913 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:28,914 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:28,914 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:28,914 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. started... 2023-11-02 03:53:28,914 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8f6ce6f475709f0e3737fe28a4b8cba4 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:28,942 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-11-02 03:53:28,945 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-11-02 03:53:28,952 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512013089d17d68b24db481789145061c5eb8 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512013089d17d68b24db481789145061c5eb8 2023-11-02 03:53:29,366 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/63545ee359e641f2aa3dde8631e252e6 2023-11-02 03:53:29,376 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/63545ee359e641f2aa3dde8631e252e6 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/63545ee359e641f2aa3dde8631e252e6 2023-11-02 03:53:29,386 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/63545ee359e641f2aa3dde8631e252e6, entries=1, sequenceid=13, filesize=5.0 K 2023-11-02 03:53:29,387 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8f6ce6f475709f0e3737fe28a4b8cba4 in 473ms, sequenceid=13, compaction requested=true 2023-11-02 03:53:29,387 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8f6ce6f475709f0e3737fe28a4b8cba4: 2023-11-02 03:53:29,387 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:29,387 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:29,388 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:29,388 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:29,388 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-02 03:53:29,388 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-02 03:53:29,390 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:29,390 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:29,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:29,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:29,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:29,391 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,391 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:29,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:29,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:29,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:29,393 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-02 03:53:29,393 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ffa1775[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:29,393 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-02 03:53:29,393 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,400 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,401 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-02 03:53:29,401 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-02 03:53:29,401 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:29,401 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:29,401 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:29,403 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:29,403 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-02 03:53:29,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:29,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:29,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:29,403 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:29,403 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-02 03:53:29,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:29,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:29,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:29,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:29,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:29,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:29,408 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-02 03:53:29,408 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-02 03:53:29,408 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:29,408 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:29,408 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,408 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:29,423 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,423 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:29,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,423 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:29,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:29,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:29,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:29,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:29,424 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:29,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:29,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:29,428 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:29,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:29,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:29,433 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:29,433 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,433 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,433 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:29,433 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,433 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,434 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,434 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:29,434 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-02 03:53:29,434 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:29,434 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:29,435 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:29,437 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:29,437 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:29,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:29,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:29,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:29,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:29,535 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:29,536 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:29,545 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:29,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:29,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,548 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-02 03:53:29,549 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:29,549 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-02 03:53:29,549 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-02 03:53:29,550 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,550 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,553 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:29,553 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:29,553 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:29,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:29,554 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:29,554 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:29,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,554 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:29,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,555 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,555 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-02 03:53:29,555 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:29,555 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-02 03:53:29,555 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:29,556 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-02 03:53:29,556 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:29,556 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:29,556 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:29,556 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. started... 2023-11-02 03:53:29,556 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8f6ce6f475709f0e3737fe28a4b8cba4 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:29,986 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120540a253eece6b42abafc84c8b4b31252c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120540a253eece6b42abafc84c8b4b31252c 2023-11-02 03:53:29,999 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/c2a21102cbcf4c1caad1c97883b73d14 2023-11-02 03:53:30,009 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/c2a21102cbcf4c1caad1c97883b73d14 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/c2a21102cbcf4c1caad1c97883b73d14 2023-11-02 03:53:30,019 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/c2a21102cbcf4c1caad1c97883b73d14, entries=1, sequenceid=17, filesize=5.0 K 2023-11-02 03:53:30,020 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8f6ce6f475709f0e3737fe28a4b8cba4 in 464ms, sequenceid=17, compaction requested=true 2023-11-02 03:53:30,020 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8f6ce6f475709f0e3737fe28a4b8cba4: 2023-11-02 03:53:30,020 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:30,021 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:30,022 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:30,022 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,022 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-02 03:53:30,022 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-02 03:53:30,024 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,024 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:30,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:30,025 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,025 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:30,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:30,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:30,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:30,028 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-02 03:53:30,028 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@11424626[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:30,028 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-02 03:53:30,028 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,029 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,030 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-02 03:53:30,030 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-02 03:53:30,030 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:30,030 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,030 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:30,032 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-02 03:53:30,032 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,032 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:30,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,032 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-02 03:53:30,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:30,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:30,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:30,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:30,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:30,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:30,037 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-02 03:53:30,037 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-02 03:53:30,037 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:30,037 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:30,037 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,037 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:30,043 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,043 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,043 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:30,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:30,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:30,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:30,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:30,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:30,045 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:30,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,047 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:30,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,052 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,052 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,052 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,052 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,052 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,052 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:30,052 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:30,052 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:30,052 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-02 03:53:30,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-02 03:53:30,053 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:30,052 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:30,052 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:30,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:30,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:30,053 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:30,054 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:30,154 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:30,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:30,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:30,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:30,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,166 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-02 03:53:30,166 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:30,167 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-02 03:53:30,167 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-02 03:53:30,167 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,167 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,170 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,170 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:30,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:30,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:30,171 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,171 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:30,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,172 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:30,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,172 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,172 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-02 03:53:30,173 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,173 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-02 03:53:30,173 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:30,173 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-02 03:53:30,173 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:30,174 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:30,174 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:30,174 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. started... 2023-11-02 03:53:30,174 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8f6ce6f475709f0e3737fe28a4b8cba4 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:30,197 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122888f486959c51450ea2ae913402e79bbb to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122888f486959c51450ea2ae913402e79bbb 2023-11-02 03:53:30,569 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-11-02 03:53:30,570 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-11-02 03:53:30,613 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/ca567edb57524449ac27b241d528e47a 2023-11-02 03:53:30,625 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/ca567edb57524449ac27b241d528e47a as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/ca567edb57524449ac27b241d528e47a 2023-11-02 03:53:30,634 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/ca567edb57524449ac27b241d528e47a, entries=1, sequenceid=21, filesize=5.0 K 2023-11-02 03:53:30,635 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8f6ce6f475709f0e3737fe28a4b8cba4 in 461ms, sequenceid=21, compaction requested=true 2023-11-02 03:53:30,636 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8f6ce6f475709f0e3737fe28a4b8cba4: 2023-11-02 03:53:30,636 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:30,636 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:30,636 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:30,636 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,636 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-02 03:53:30,637 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-02 03:53:30,639 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,639 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:30,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:30,640 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,640 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:30,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:30,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:30,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:30,642 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-02 03:53:30,642 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-02 03:53:30,642 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4155a853[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:30,642 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,644 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,644 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-02 03:53:30,644 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-02 03:53:30,644 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:30,644 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,644 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:30,646 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-02 03:53:30,646 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:30,646 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,646 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-02 03:53:30,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:30,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:30,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:30,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:30,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:30,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:30,650 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-02 03:53:30,650 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-02 03:53:30,651 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:30,651 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:30,651 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,651 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:30,653 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,653 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:30,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,653 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:30,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:30,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:30,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:30,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:30,655 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:30,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,656 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:30,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,660 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,660 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:30,660 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:30,660 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:30,660 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:30,660 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,660 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,660 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,660 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,660 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:30,660 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-02 03:53:30,661 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:30,661 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-02 03:53:30,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:30,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:30,661 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:30,661 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:30,762 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:30,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:30,771 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:30,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:30,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,774 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-02 03:53:30,774 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:30,775 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-02 03:53:30,775 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-02 03:53:30,776 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,776 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,780 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:30,780 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:30,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:30,781 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,781 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:30,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,781 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:30,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,782 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,782 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-02 03:53:30,782 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,782 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-02 03:53:30,782 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:30,783 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-02 03:53:30,783 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:30,783 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:30,783 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:30,783 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. started... 2023-11-02 03:53:30,783 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8f6ce6f475709f0e3737fe28a4b8cba4 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:30,804 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512311992250040ff4cb395e71da29cd62b50 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512311992250040ff4cb395e71da29cd62b50 2023-11-02 03:53:30,850 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/0a72ea2903b5454ba4d1e5006743d253 2023-11-02 03:53:30,859 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/0a72ea2903b5454ba4d1e5006743d253 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/0a72ea2903b5454ba4d1e5006743d253 2023-11-02 03:53:30,867 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/0a72ea2903b5454ba4d1e5006743d253, entries=1, sequenceid=25, filesize=5.0 K 2023-11-02 03:53:30,869 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8f6ce6f475709f0e3737fe28a4b8cba4 in 86ms, sequenceid=25, compaction requested=true 2023-11-02 03:53:30,869 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8f6ce6f475709f0e3737fe28a4b8cba4: 2023-11-02 03:53:30,869 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:30,869 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:30,869 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:30,869 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,869 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-02 03:53:30,869 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-02 03:53:30,871 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,871 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:30,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:30,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:30,872 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,872 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:30,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:30,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:30,875 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-02 03:53:30,875 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5d3e1435[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:30,875 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-02 03:53:30,875 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,877 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,877 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-02 03:53:30,877 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-02 03:53:30,877 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:30,877 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,877 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:30,879 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-02 03:53:30,879 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,879 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:30,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:30,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:30,879 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-02 03:53:30,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:30,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:30,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:30,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:30,884 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-02 03:53:30,884 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-02 03:53:30,884 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:30,884 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:30,884 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,884 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:30,887 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,887 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,887 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:30,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:30,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:30,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:30,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:30,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:30,888 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:30,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,890 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:30,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,893 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,893 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,893 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,893 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:30,893 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:30,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:30,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:30,894 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:30,894 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-02 03:53:30,894 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-02 03:53:30,894 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:30,894 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,894 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:30,895 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:30,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:30,895 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:30,895 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:30,895 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:30,995 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:30,997 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:31,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:31,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:31,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,007 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-02 03:53:31,007 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:31,008 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-02 03:53:31,008 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-02 03:53:31,009 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,009 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,011 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,011 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:31,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:31,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:31,011 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,011 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:31,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,012 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:31,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,013 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,013 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-02 03:53:31,013 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,013 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-02 03:53:31,013 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:31,014 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-02 03:53:31,014 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:31,014 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:31,014 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:31,014 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. started... 2023-11-02 03:53:31,014 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8f6ce6f475709f0e3737fe28a4b8cba4 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:31,037 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010129d6f32280c1462da916f4c54769dba7 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010129d6f32280c1462da916f4c54769dba7 2023-11-02 03:53:31,051 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/c4a55eda2902406b8025677e004eb168 2023-11-02 03:53:31,074 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/c4a55eda2902406b8025677e004eb168 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/c4a55eda2902406b8025677e004eb168 2023-11-02 03:53:31,102 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/c4a55eda2902406b8025677e004eb168, entries=1, sequenceid=29, filesize=5.0 K 2023-11-02 03:53:31,103 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8f6ce6f475709f0e3737fe28a4b8cba4 in 89ms, sequenceid=29, compaction requested=true 2023-11-02 03:53:31,103 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8f6ce6f475709f0e3737fe28a4b8cba4: 2023-11-02 03:53:31,104 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:31,104 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:31,104 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:31,104 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,104 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-02 03:53:31,104 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-02 03:53:31,111 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,111 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:31,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:31,112 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,112 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:31,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:31,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:31,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:31,114 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-02 03:53:31,114 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@52f0e158[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:31,114 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-02 03:53:31,114 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,116 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,116 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-02 03:53:31,116 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-02 03:53:31,116 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:31,116 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,116 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:31,118 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-02 03:53:31,118 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,118 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:31,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:31,118 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-02 03:53:31,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:31,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:31,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:31,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:31,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:31,122 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-02 03:53:31,122 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-02 03:53:31,123 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:31,123 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:31,123 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,123 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:31,125 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,125 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:31,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:31,125 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:31,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:31,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:31,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:31,126 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:31,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,128 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:31,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,132 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,133 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,133 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,133 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,133 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,133 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,133 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:31,133 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:31,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:31,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:31,133 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,133 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:31,134 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-02 03:53:31,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:31,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:31,134 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:31,135 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:31,135 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:31,235 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:31,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:31,247 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:31,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:31,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,250 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-02 03:53:31,250 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:31,250 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-02 03:53:31,250 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-02 03:53:31,251 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,251 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,253 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,253 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:31,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:31,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:31,254 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,254 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:31,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,255 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:31,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,255 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,255 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-02 03:53:31,255 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,256 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-02 03:53:31,256 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:31,256 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-02 03:53:31,256 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:31,256 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:31,256 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:31,256 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. started... 2023-11-02 03:53:31,257 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8f6ce6f475709f0e3737fe28a4b8cba4 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:31,279 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103443809b29b5549b8868550482a766612 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103443809b29b5549b8868550482a766612 2023-11-02 03:53:31,691 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/521e092d06d64e6390ef2dadc1671e1b 2023-11-02 03:53:31,701 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/521e092d06d64e6390ef2dadc1671e1b as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/521e092d06d64e6390ef2dadc1671e1b 2023-11-02 03:53:31,710 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/521e092d06d64e6390ef2dadc1671e1b, entries=1, sequenceid=33, filesize=5.0 K 2023-11-02 03:53:31,711 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8f6ce6f475709f0e3737fe28a4b8cba4 in 454ms, sequenceid=33, compaction requested=true 2023-11-02 03:53:31,711 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8f6ce6f475709f0e3737fe28a4b8cba4: 2023-11-02 03:53:31,711 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:31,711 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:31,712 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:31,712 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,712 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-02 03:53:31,712 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-02 03:53:31,715 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:31,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:31,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:31,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:31,716 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,716 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,716 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:31,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:31,718 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-02 03:53:31,718 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5180fe56[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:31,718 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-02 03:53:31,718 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,720 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,720 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:31,722 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,722 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,722 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-02 03:53:31,722 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-02 03:53:31,722 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:31,725 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,725 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-02 03:53:31,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:31,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:31,725 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:31,725 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-02 03:53:31,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:31,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:31,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:31,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:31,730 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-02 03:53:31,730 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-02 03:53:31,730 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:31,730 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:31,730 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,730 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:31,735 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,735 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:31,735 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:31,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:31,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:31,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:31,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:31,736 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:31,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,738 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:31,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,742 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,742 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:31,742 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:31,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:31,742 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,742 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-02 03:53:31,742 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:31,742 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-02 03:53:31,742 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:31,742 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:31,742 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,742 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,742 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,743 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:31,743 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:31,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:31,743 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:31,844 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:31,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:31,861 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:31,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:31,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,863 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-02 03:53:31,864 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:31,864 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-02 03:53:31,864 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-02 03:53:31,865 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,865 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,867 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,867 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:31,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:31,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:31,868 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:31,868 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:31,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,869 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:31,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,869 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,870 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-02 03:53:31,871 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:31,871 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-02 03:53:31,871 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:31,871 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-02 03:53:31,872 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:31,872 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:31,872 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:31,872 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. started... 2023-11-02 03:53:31,872 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8f6ce6f475709f0e3737fe28a4b8cba4 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:31,913 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907fc285dbf251d4b6999bc3800a7045818 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20150907fc285dbf251d4b6999bc3800a7045818 2023-11-02 03:53:32,331 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/e2fab6354d57463d80b94b341124fedc 2023-11-02 03:53:32,340 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/e2fab6354d57463d80b94b341124fedc as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/e2fab6354d57463d80b94b341124fedc 2023-11-02 03:53:32,349 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/e2fab6354d57463d80b94b341124fedc, entries=1, sequenceid=37, filesize=5.0 K 2023-11-02 03:53:32,350 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8f6ce6f475709f0e3737fe28a4b8cba4 in 478ms, sequenceid=37, compaction requested=true 2023-11-02 03:53:32,350 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8f6ce6f475709f0e3737fe28a4b8cba4: 2023-11-02 03:53:32,350 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:32,350 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:32,351 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:32,351 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,351 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-02 03:53:32,351 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-02 03:53:32,354 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,354 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:32,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:32,355 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,355 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:32,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:32,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:32,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:32,357 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-02 03:53:32,357 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3d7bc305[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:32,357 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-02 03:53:32,357 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,359 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,359 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-02 03:53:32,359 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-02 03:53:32,359 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:32,359 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,359 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:32,361 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-02 03:53:32,361 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,361 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:32,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,361 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-02 03:53:32,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:32,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:32,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:32,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:32,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:32,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:32,366 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-02 03:53:32,366 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-02 03:53:32,366 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:32,366 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:32,366 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,366 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:32,368 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,368 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:32,368 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:32,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:32,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:32,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:32,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:32,370 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:32,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,372 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:32,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,381 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:32,381 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:32,381 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,381 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:32,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:32,381 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,381 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:32,381 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,382 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-02 03:53:32,382 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:32,382 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,383 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,383 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,383 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:32,383 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:32,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:32,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:32,483 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:32,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:32,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:32,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:32,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,497 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-02 03:53:32,497 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:32,497 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-11-02 03:53:32,498 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-11-02 03:53:32,498 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,498 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,500 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,500 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:32,500 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:32,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:32,501 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,501 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:32,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,502 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:32,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,502 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,503 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-11-02 03:53:32,503 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,503 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-11-02 03:53:32,503 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:32,504 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-11-02 03:53:32,504 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:32,504 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:32,504 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:32,504 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. started... 2023-11-02 03:53:32,505 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 8f6ce6f475709f0e3737fe28a4b8cba4 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:32,528 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120342058b24875490aae23b2ab692eb69c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120342058b24875490aae23b2ab692eb69c 2023-11-02 03:53:32,538 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/8246c8decb6c41c0bc0f3da16eb6d5d3 2023-11-02 03:53:32,545 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/8246c8decb6c41c0bc0f3da16eb6d5d3 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/8246c8decb6c41c0bc0f3da16eb6d5d3 2023-11-02 03:53:32,552 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/8246c8decb6c41c0bc0f3da16eb6d5d3, entries=1, sequenceid=41, filesize=5.0 K 2023-11-02 03:53:32,553 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8f6ce6f475709f0e3737fe28a4b8cba4 in 48ms, sequenceid=41, compaction requested=true 2023-11-02 03:53:32,553 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 8f6ce6f475709f0e3737fe28a4b8cba4: 2023-11-02 03:53:32,553 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:32,553 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:32,554 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:32,554 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,554 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-11-02 03:53:32,554 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-11-02 03:53:32,556 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,556 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:32,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:32,556 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,556 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:32,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:32,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:32,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:32,558 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-11-02 03:53:32,558 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1049e723[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:32,558 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-11-02 03:53:32,558 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,563 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,564 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-11-02 03:53:32,564 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-11-02 03:53:32,564 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:32,564 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,564 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:32,565 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-11-02 03:53:32,565 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,565 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:32,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:32,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:32,566 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-11-02 03:53:32,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:32,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:32,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:32,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:32,570 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-11-02 03:53:32,570 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-11-02 03:53:32,570 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:32,571 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:32,571 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,571 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:32,573 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,573 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:32,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:32,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,573 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:32,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:32,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:32,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:32,573 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:32,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,575 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:32,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,579 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:32,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:32,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:32,579 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:32,579 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-11-02 03:53:32,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:32,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:32,581 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,581 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,581 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,581 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:32,581 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,581 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,582 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,582 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:32,582 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-11-02 03:53:32,582 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:32,582 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:32,582 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:32,682 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:32,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:32,691 DEBUG [Listener at localhost/43647] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-11-02 03:53:32,693 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49310, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-11-02 03:53:32,695 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41275] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:32,708 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] procedure2.ProcedureExecutor(1028): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-02 03:53:32,709 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-02 03:53:32,711 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:53:32,711 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698897812711 2023-11-02 03:53:32,713 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-02 03:53:32,718 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-11-02 03:53:32,718 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-02 03:53:32,720 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] zookeeper.ReadOnlyZKClient(139): Connect 0x6b262b5d to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:53:32,726 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2c519d79, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:53:32,728 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-02 03:53:33,208 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010389b188803c334e9a83b501e090247e1d to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010389b188803c334e9a83b501e090247e1d 2023-11-02 03:53:33,209 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-02 03:53:33,228 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-02 03:53:33,230 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41218, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-02 03:53:33,241 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-02 03:53:33,243 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49324, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-02 03:53:33,278 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/614a658b7137437ba6d5dbc163c2871c first=Optional[r3] last=Optional[r6] 2023-11-02 03:53:33,291 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/614a658b7137437ba6d5dbc163c2871c}] 2023-11-02 03:53:33,308 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/614a658b7137437ba6d5dbc163c2871c for inclusion in 8f6ce6f475709f0e3737fe28a4b8cba4/family1 2023-11-02 03:53:33,315 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-11-02 03:53:33,315 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last= 2023-11-02 03:53:33,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for 8f6ce6f475709f0e3737fe28a4b8cba4: 2023-11-02 03:53:33,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/614a658b7137437ba6d5dbc163c2871c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__s8ehhdq0upahq7ttbfaftjofi5kl3vtebmn65tpeop0v753r1nh0n5d28bsucvsj/family1/614a658b7137437ba6d5dbc163c2871c 2023-11-02 03:53:33,321 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__s8ehhdq0upahq7ttbfaftjofi5kl3vtebmn65tpeop0v753r1nh0n5d28bsucvsj/family1/614a658b7137437ba6d5dbc163c2871c as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/8d4d05ed160b40d3939889c506fddf95_SeqId_44_ 2023-11-02 03:53:33,322 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__s8ehhdq0upahq7ttbfaftjofi5kl3vtebmn65tpeop0v753r1nh0n5d28bsucvsj/family1/614a658b7137437ba6d5dbc163c2871c into 8f6ce6f475709f0e3737fe28a4b8cba4/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/8d4d05ed160b40d3939889c506fddf95_SeqId_44_ - updating store file list. 2023-11-02 03:53:33,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/8d4d05ed160b40d3939889c506fddf95_SeqId_44_ into 8f6ce6f475709f0e3737fe28a4b8cba4/family1 2023-11-02 03:53:33,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__s8ehhdq0upahq7ttbfaftjofi5kl3vtebmn65tpeop0v753r1nh0n5d28bsucvsj/family1/614a658b7137437ba6d5dbc163c2871c into 8f6ce6f475709f0e3737fe28a4b8cba4/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/8d4d05ed160b40d3939889c506fddf95_SeqId_44_) 2023-11-02 03:53:33,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__s8ehhdq0upahq7ttbfaftjofi5kl3vtebmn65tpeop0v753r1nh0n5d28bsucvsj/family1/614a658b7137437ba6d5dbc163c2871c 2023-11-02 03:53:33,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-02 03:53:33,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-02 03:53:33,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(1912): 8f6ce6f475709f0e3737fe28a4b8cba4/family1 is initiating minor compaction 2023-11-02 03:53:33,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@44ce3d3b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:53:33,342 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 8f6ce6f475709f0e3737fe28a4b8cba4/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:33,342 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/3fb8ad9e6b4545d59274b9ab6a39eb8d, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/63545ee359e641f2aa3dde8631e252e6, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/c2a21102cbcf4c1caad1c97883b73d14, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/ca567edb57524449ac27b241d528e47a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/0a72ea2903b5454ba4d1e5006743d253, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/c4a55eda2902406b8025677e004eb168, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/521e092d06d64e6390ef2dadc1671e1b, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/e2fab6354d57463d80b94b341124fedc, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/8246c8decb6c41c0bc0f3da16eb6d5d3, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/8d4d05ed160b40d3939889c506fddf95_SeqId_44_] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp, totalSize=50.2 K 2023-11-02 03:53:33,343 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 3fb8ad9e6b4545d59274b9ab6a39eb8d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-02 03:53:33,343 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 63545ee359e641f2aa3dde8631e252e6, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-02 03:53:33,343 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting c2a21102cbcf4c1caad1c97883b73d14, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-02 03:53:33,343 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting ca567edb57524449ac27b241d528e47a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-02 03:53:33,343 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 0a72ea2903b5454ba4d1e5006743d253, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-02 03:53:33,343 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting c4a55eda2902406b8025677e004eb168, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-02 03:53:33,343 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 521e092d06d64e6390ef2dadc1671e1b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-02 03:53:33,343 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting e2fab6354d57463d80b94b341124fedc, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-02 03:53:33,344 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 8246c8decb6c41c0bc0f3da16eb6d5d3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-02 03:53:33,344 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 8d4d05ed160b40d3939889c506fddf95_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-02 03:53:33,357 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-02 03:53:33,359 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:53:33,365 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122888f486959c51450ea2ae913402e79bbb to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015122888f486959c51450ea2ae913402e79bbb 2023-11-02 03:53:33,367 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512311992250040ff4cb395e71da29cd62b50 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512311992250040ff4cb395e71da29cd62b50 2023-11-02 03:53:33,369 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010129d6f32280c1462da916f4c54769dba7 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010129d6f32280c1462da916f4c54769dba7 2023-11-02 03:53:33,371 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103443809b29b5549b8868550482a766612 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103443809b29b5549b8868550482a766612 2023-11-02 03:53:33,371 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-11-02 03:53:33,371 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-11-02 03:53:33,371 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-02 03:53:33,372 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-02 03:53:33,372 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-02 03:53:33,372 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-02 03:53:33,372 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-02 03:53:33,373 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-02 03:53:33,373 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-02 03:53:33,373 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-02 03:53:33,406 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8f6ce6f475709f0e3737fe28a4b8cba4#family1#compaction#10 average throughput is 0.36 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:53:33,841 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/4c844c96ac704f2fa9b7ba717ad23ec3 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/4c844c96ac704f2fa9b7ba717ad23ec3 2023-11-02 03:53:33,860 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 8f6ce6f475709f0e3737fe28a4b8cba4/family1 of 8f6ce6f475709f0e3737fe28a4b8cba4 into 4c844c96ac704f2fa9b7ba717ad23ec3(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-02 03:53:33,861 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8f6ce6f475709f0e3737fe28a4b8cba4: 2023-11-02 03:53:33,866 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1698897213340; duration=0sec 2023-11-02 03:53:33,867 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:53:33,870 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/3fb8ad9e6b4545d59274b9ab6a39eb8d, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/63545ee359e641f2aa3dde8631e252e6, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/c2a21102cbcf4c1caad1c97883b73d14, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/ca567edb57524449ac27b241d528e47a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/0a72ea2903b5454ba4d1e5006743d253, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/c4a55eda2902406b8025677e004eb168, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/521e092d06d64e6390ef2dadc1671e1b, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/e2fab6354d57463d80b94b341124fedc, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/8246c8decb6c41c0bc0f3da16eb6d5d3, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/8d4d05ed160b40d3939889c506fddf95_SeqId_44_] to archive 2023-11-02 03:53:33,872 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:53:33,875 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/3fb8ad9e6b4545d59274b9ab6a39eb8d to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/3fb8ad9e6b4545d59274b9ab6a39eb8d 2023-11-02 03:53:33,878 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/63545ee359e641f2aa3dde8631e252e6 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/63545ee359e641f2aa3dde8631e252e6 2023-11-02 03:53:33,880 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/c2a21102cbcf4c1caad1c97883b73d14 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/c2a21102cbcf4c1caad1c97883b73d14 2023-11-02 03:53:33,887 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/ca567edb57524449ac27b241d528e47a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/ca567edb57524449ac27b241d528e47a 2023-11-02 03:53:33,889 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/0a72ea2903b5454ba4d1e5006743d253 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/0a72ea2903b5454ba4d1e5006743d253 2023-11-02 03:53:33,891 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/c4a55eda2902406b8025677e004eb168 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/c4a55eda2902406b8025677e004eb168 2023-11-02 03:53:33,892 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/521e092d06d64e6390ef2dadc1671e1b to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/521e092d06d64e6390ef2dadc1671e1b 2023-11-02 03:53:33,894 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/e2fab6354d57463d80b94b341124fedc to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/e2fab6354d57463d80b94b341124fedc 2023-11-02 03:53:33,896 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/8246c8decb6c41c0bc0f3da16eb6d5d3 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/8246c8decb6c41c0bc0f3da16eb6d5d3 2023-11-02 03:53:33,898 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/8d4d05ed160b40d3939889c506fddf95_SeqId_44_ to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/8d4d05ed160b40d3939889c506fddf95_SeqId_44_ 2023-11-02 03:53:34,293 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205322441e3c03f480a94acb9615a838c8e to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205322441e3c03f480a94acb9615a838c8e 2023-11-02 03:53:34,295 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-02 03:53:34,334 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9e1331df12374ebe8b3c11be24d53166 first=Optional[r00] last=Optional[r2] 2023-11-02 03:53:34,335 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9e1331df12374ebe8b3c11be24d53166}] 2023-11-02 03:53:34,337 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9e1331df12374ebe8b3c11be24d53166 for inclusion in 8f6ce6f475709f0e3737fe28a4b8cba4/family1 2023-11-02 03:53:34,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-11-02 03:53:34,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last= 2023-11-02 03:53:34,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for 8f6ce6f475709f0e3737fe28a4b8cba4: 2023-11-02 03:53:34,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9e1331df12374ebe8b3c11be24d53166 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__k5cq6fiflpk82442om2qd4vlj6q1hbj7fuau9njfnh97q7f54h8g8i9j4p2mbdj/family1/9e1331df12374ebe8b3c11be24d53166 2023-11-02 03:53:34,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__k5cq6fiflpk82442om2qd4vlj6q1hbj7fuau9njfnh97q7f54h8g8i9j4p2mbdj/family1/9e1331df12374ebe8b3c11be24d53166 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/d18e0d9dc15b445881fac62bbf00843c_SeqId_47_ 2023-11-02 03:53:34,347 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__k5cq6fiflpk82442om2qd4vlj6q1hbj7fuau9njfnh97q7f54h8g8i9j4p2mbdj/family1/9e1331df12374ebe8b3c11be24d53166 into 8f6ce6f475709f0e3737fe28a4b8cba4/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/d18e0d9dc15b445881fac62bbf00843c_SeqId_47_ - updating store file list. 2023-11-02 03:53:34,353 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/d18e0d9dc15b445881fac62bbf00843c_SeqId_47_ into 8f6ce6f475709f0e3737fe28a4b8cba4/family1 2023-11-02 03:53:34,353 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__k5cq6fiflpk82442om2qd4vlj6q1hbj7fuau9njfnh97q7f54h8g8i9j4p2mbdj/family1/9e1331df12374ebe8b3c11be24d53166 into 8f6ce6f475709f0e3737fe28a4b8cba4/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/d18e0d9dc15b445881fac62bbf00843c_SeqId_47_) 2023-11-02 03:53:34,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__k5cq6fiflpk82442om2qd4vlj6q1hbj7fuau9njfnh97q7f54h8g8i9j4p2mbdj/family1/9e1331df12374ebe8b3c11be24d53166 2023-11-02 03:53:34,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-02 03:53:34,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-02 03:53:34,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(1912): 8f6ce6f475709f0e3737fe28a4b8cba4/family1 is initiating minor compaction (all files) 2023-11-02 03:53:34,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@602882e0; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:53:34,356 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 8f6ce6f475709f0e3737fe28a4b8cba4/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:34,357 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/4f5bf6d28d6846c79d59ecfa56cf78fb, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/4c844c96ac704f2fa9b7ba717ad23ec3, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/d18e0d9dc15b445881fac62bbf00843c_SeqId_47_] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp, totalSize=358.6 K 2023-11-02 03:53:34,365 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 4f5bf6d28d6846c79d59ecfa56cf78fb, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878824741 2023-11-02 03:53:34,366 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 4c844c96ac704f2fa9b7ba717ad23ec3, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621224741 2023-11-02 03:53:34,366 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting d18e0d9dc15b445881fac62bbf00843c_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878824741 2023-11-02 03:53:34,370 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-02 03:53:34,372 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:53:34,376 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113073c5250cf4124493ac94e5467bcac57d to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015113073c5250cf4124493ac94e5467bcac57d 2023-11-02 03:53:34,378 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512013089d17d68b24db481789145061c5eb8 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512013089d17d68b24db481789145061c5eb8 2023-11-02 03:53:34,381 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120540a253eece6b42abafc84c8b4b31252c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120540a253eece6b42abafc84c8b4b31252c 2023-11-02 03:53:34,381 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-11-02 03:53:34,381 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-11-02 03:53:34,382 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-02 03:53:34,382 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-02 03:53:34,383 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:53:34,384 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-02 03:53:34,387 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 1.6840 sec 2023-11-02 03:53:34,419 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8f6ce6f475709f0e3737fe28a4b8cba4#family1#compaction#11 average throughput is 4.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:53:34,578 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-11-02 03:53:34,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:35,005 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-11-02 03:53:35,008 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-11-02 03:53:35,008 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-11-02 03:53:35,012 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-11-02 03:53:35,025 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=8f6ce6f475709f0e3737fe28a4b8cba4, REOPEN/MOVE}] 2023-11-02 03:53:35,027 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=8f6ce6f475709f0e3737fe28a4b8cba4, REOPEN/MOVE 2023-11-02 03:53:35,028 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=8f6ce6f475709f0e3737fe28a4b8cba4, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:35,029 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1698897215028"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897215028"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897215028"}]},"ts":"1698897215028"} 2023-11-02 03:53:35,031 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure 8f6ce6f475709f0e3737fe28a4b8cba4, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:53:35,189 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:35,191 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 8f6ce6f475709f0e3737fe28a4b8cba4 2023-11-02 03:53:35,191 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8f6ce6f475709f0e3737fe28a4b8cba4, disabling compactions & flushes 2023-11-02 03:53:35,191 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:35,662 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/.tmp/family1/1b4d48fef03a4458b7b547d611bec6ef as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/1b4d48fef03a4458b7b547d611bec6ef 2023-11-02 03:53:35,670 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 8f6ce6f475709f0e3737fe28a4b8cba4/family1 of 8f6ce6f475709f0e3737fe28a4b8cba4 into 1b4d48fef03a4458b7b547d611bec6ef(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-02 03:53:35,671 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8f6ce6f475709f0e3737fe28a4b8cba4: 2023-11-02 03:53:35,671 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:35,671 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:35,671 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. after waiting 0 ms 2023-11-02 03:53:35,671 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1698897214356; duration=1sec 2023-11-02 03:53:35,671 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:35,671 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:53:35,675 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/4f5bf6d28d6846c79d59ecfa56cf78fb, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/4c844c96ac704f2fa9b7ba717ad23ec3, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/d18e0d9dc15b445881fac62bbf00843c_SeqId_47_] to archive 2023-11-02 03:53:35,676 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:53:35,678 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/4f5bf6d28d6846c79d59ecfa56cf78fb to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/4f5bf6d28d6846c79d59ecfa56cf78fb 2023-11-02 03:53:35,680 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/4c844c96ac704f2fa9b7ba717ad23ec3 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/4c844c96ac704f2fa9b7ba717ad23ec3 2023-11-02 03:53:35,682 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/d18e0d9dc15b445881fac62bbf00843c_SeqId_47_ to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/d18e0d9dc15b445881fac62bbf00843c_SeqId_47_ 2023-11-02 03:53:35,689 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-11-02 03:53:35,691 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:35,691 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8f6ce6f475709f0e3737fe28a4b8cba4: 2023-11-02 03:53:35,691 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3510): Not adding moved region record: 8f6ce6f475709f0e3737fe28a4b8cba4 to self. 2023-11-02 03:53:35,694 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 8f6ce6f475709f0e3737fe28a4b8cba4 2023-11-02 03:53:35,694 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=8f6ce6f475709f0e3737fe28a4b8cba4, regionState=CLOSED 2023-11-02 03:53:35,695 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1698897215694"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897215694"}]},"ts":"1698897215694"} 2023-11-02 03:53:35,699 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=16, resume processing ppid=15 2023-11-02 03:53:35,700 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure 8f6ce6f475709f0e3737fe28a4b8cba4, server=jenkins-hbase4.apache.org,36627,1698897201574 in 666 msec 2023-11-02 03:53:35,701 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=8f6ce6f475709f0e3737fe28a4b8cba4, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=true 2023-11-02 03:53:35,851 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=8f6ce6f475709f0e3737fe28a4b8cba4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:35,852 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1698897215851"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897215851"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897215851"}]},"ts":"1698897215851"} 2023-11-02 03:53:35,855 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 8f6ce6f475709f0e3737fe28a4b8cba4, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:53:36,007 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:36,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:36,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8f6ce6f475709f0e3737fe28a4b8cba4, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4.', STARTKEY => '', ENDKEY => ''} 2023-11-02 03:53:36,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 8f6ce6f475709f0e3737fe28a4b8cba4 2023-11-02 03:53:36,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:53:36,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8f6ce6f475709f0e3737fe28a4b8cba4 2023-11-02 03:53:36,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8f6ce6f475709f0e3737fe28a4b8cba4 2023-11-02 03:53:36,014 INFO [StoreOpener-8f6ce6f475709f0e3737fe28a4b8cba4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8f6ce6f475709f0e3737fe28a4b8cba4 2023-11-02 03:53:36,015 DEBUG [StoreOpener-8f6ce6f475709f0e3737fe28a4b8cba4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1 2023-11-02 03:53:36,015 DEBUG [StoreOpener-8f6ce6f475709f0e3737fe28a4b8cba4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1 2023-11-02 03:53:36,016 INFO [StoreOpener-8f6ce6f475709f0e3737fe28a4b8cba4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8f6ce6f475709f0e3737fe28a4b8cba4 columnFamilyName family1 2023-11-02 03:53:36,023 DEBUG [StoreOpener-8f6ce6f475709f0e3737fe28a4b8cba4-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/1b4d48fef03a4458b7b547d611bec6ef 2023-11-02 03:53:36,023 INFO [StoreOpener-8f6ce6f475709f0e3737fe28a4b8cba4-1] regionserver.HStore(310): Store=8f6ce6f475709f0e3737fe28a4b8cba4/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:53:36,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4 2023-11-02 03:53:36,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4 2023-11-02 03:53:36,029 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8f6ce6f475709f0e3737fe28a4b8cba4 2023-11-02 03:53:36,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8f6ce6f475709f0e3737fe28a4b8cba4; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73476312, jitterRate=0.09488236904144287}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-02 03:53:36,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8f6ce6f475709f0e3737fe28a4b8cba4: 2023-11-02 03:53:36,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4., pid=17, masterSystemTime=1698897216007 2023-11-02 03:53:36,033 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:36,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:53:36,034 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=8f6ce6f475709f0e3737fe28a4b8cba4, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:36,034 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1698897216033"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897216033"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897216033"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897216033"}]},"ts":"1698897216033"} 2023-11-02 03:53:36,108 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=17, resume processing ppid=15 2023-11-02 03:53:36,108 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 8f6ce6f475709f0e3737fe28a4b8cba4, server=jenkins-hbase4.apache.org,36627,1698897201574 in 181 msec 2023-11-02 03:53:36,110 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=15, resume processing ppid=14 2023-11-02 03:53:36,110 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=8f6ce6f475709f0e3737fe28a4b8cba4, REOPEN/MOVE in 1.0830 sec 2023-11-02 03:53:36,114 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=14, resume processing ppid=13 2023-11-02 03:53:36,114 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.1010 sec 2023-11-02 03:53:36,116 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.5340 sec 2023-11-02 03:53:36,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=13 2023-11-02 03:53:36,126 INFO [Listener at localhost/43647] compactions.TestMobCompactor(278): alter status finished 2023-11-02 03:53:36,127 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41275] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-11-02 03:53:36,129 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] procedure2.ProcedureExecutor(1028): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-02 03:53:36,130 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-02 03:53:36,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:53:36,130 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698897816130 2023-11-02 03:53:36,132 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-02 03:53:36,132 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-11-02 03:53:36,133 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-02 03:53:36,134 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] zookeeper.ReadOnlyZKClient(139): Connect 0x37eaf16e to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:53:36,190 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1d26809f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:53:36,190 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-02 03:53:36,190 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-02 03:53:36,191 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-02 03:53:36,191 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-02 03:53:36,191 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-02 03:53:36,627 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112863a8b47e0dec45bcad8663e56135da39 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112863a8b47e0dec45bcad8663e56135da39 2023-11-02 03:53:36,628 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-02 03:53:36,642 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-02 03:53:36,643 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48394, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-02 03:53:36,652 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-02 03:53:36,654 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59980, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-02 03:53:36,666 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/907d576e2d2f4ec58750cf6477a3c3b9 first=Optional[r05] last=Optional[r8] 2023-11-02 03:53:36,667 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=51 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/907d576e2d2f4ec58750cf6477a3c3b9}] 2023-11-02 03:53:36,672 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/907d576e2d2f4ec58750cf6477a3c3b9 for inclusion in 8f6ce6f475709f0e3737fe28a4b8cba4/family1 2023-11-02 03:53:36,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-11-02 03:53:36,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last= 2023-11-02 03:53:36,675 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for 8f6ce6f475709f0e3737fe28a4b8cba4: 2023-11-02 03:53:36,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/907d576e2d2f4ec58750cf6477a3c3b9 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ibhi0d25a03qmecfeec3rn1k7k9f51o6dmdtinfdcui4h44veem3v4u777spne5l/family1/907d576e2d2f4ec58750cf6477a3c3b9 2023-11-02 03:53:36,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ibhi0d25a03qmecfeec3rn1k7k9f51o6dmdtinfdcui4h44veem3v4u777spne5l/family1/907d576e2d2f4ec58750cf6477a3c3b9 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/aef35934f9924916a5e9b94d172cef1c_SeqId_53_ 2023-11-02 03:53:36,680 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ibhi0d25a03qmecfeec3rn1k7k9f51o6dmdtinfdcui4h44veem3v4u777spne5l/family1/907d576e2d2f4ec58750cf6477a3c3b9 into 8f6ce6f475709f0e3737fe28a4b8cba4/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/aef35934f9924916a5e9b94d172cef1c_SeqId_53_ - updating store file list. 2023-11-02 03:53:36,686 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/aef35934f9924916a5e9b94d172cef1c_SeqId_53_ into 8f6ce6f475709f0e3737fe28a4b8cba4/family1 2023-11-02 03:53:36,686 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ibhi0d25a03qmecfeec3rn1k7k9f51o6dmdtinfdcui4h44veem3v4u777spne5l/family1/907d576e2d2f4ec58750cf6477a3c3b9 into 8f6ce6f475709f0e3737fe28a4b8cba4/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/family1/aef35934f9924916a5e9b94d172cef1c_SeqId_53_) 2023-11-02 03:53:36,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__ibhi0d25a03qmecfeec3rn1k7k9f51o6dmdtinfdcui4h44veem3v4u777spne5l/family1/907d576e2d2f4ec58750cf6477a3c3b9 2023-11-02 03:53:36,689 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-02 03:53:36,690 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:53:36,692 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120342058b24875490aae23b2ab692eb69c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120342058b24875490aae23b2ab692eb69c 2023-11-02 03:53:36,693 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128da3232e95eae424db2b22dc9ae696f07 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128da3232e95eae424db2b22dc9ae696f07 2023-11-02 03:53:36,693 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-11-02 03:53:36,693 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-11-02 03:53:36,694 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-02 03:53:36,694 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-02 03:53:36,694 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-11-02 03:53:36,694 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-11-02 03:53:36,694 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:53:36,695 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-11-02 03:53:36,697 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 567 msec 2023-11-02 03:53:36,798 INFO [Listener at localhost/43647] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=252 (was 214) Potentially hanging thread: RS:0;jenkins-hbase4:36627-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:54550 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:54798 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:54606 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1dd9e7a2-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:54654 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1105563298_17 at /127.0.0.1:54772 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:54664 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65185@0x37eaf16e-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:549) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:54526 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65185@0x37eaf16e-SendThread(127.0.0.1:65185) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:332) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1289) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65185@0x6b262b5d-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:549) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:54724 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:316) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:373) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x1dd9e7a2-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:54784 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:54640 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:54826 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xa24eafb-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65185@0x6b262b5d sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$88/1750819181.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:54624 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xa24eafb-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:54680 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:54750 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:54500 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:54588 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65185@0x6b262b5d-SendThread(127.0.0.1:65185) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:332) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1289) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:54458 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:54708 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xa24eafb-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65185@0x37eaf16e sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$88/1750819181.run(Unknown Source) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=636 (was 505) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=352 (was 360), ProcessCount=170 (was 170), AvailableMemoryMB=11792 (was 11664) - AvailableMemoryMB LEAK? - 2023-11-02 03:53:36,816 INFO [Listener at localhost/43647] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=252, OpenFileDescriptor=636, MaxFileDescriptor=60000, SystemLoadAverage=352, ProcessCount=170, AvailableMemoryMB=11790 2023-11-02 03:53:36,819 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-11-02 03:53:36,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:36,822 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-11-02 03:53:36,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-11-02 03:53:36,823 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-02 03:53:36,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-02 03:53:36,825 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de 2023-11-02 03:53:36,826 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de empty. 2023-11-02 03:53:36,827 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de 2023-11-02 03:53:36,827 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-11-02 03:53:36,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-02 03:53:37,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-02 03:53:37,242 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-11-02 03:53:37,243 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => c21eb15d0957cd8a2c44a1d684d8b7de, NAME => 'testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp 2023-11-02 03:53:37,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-02 03:53:37,654 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:53:37,654 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing c21eb15d0957cd8a2c44a1d684d8b7de, disabling compactions & flushes 2023-11-02 03:53:37,654 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:37,655 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:37,655 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. after waiting 0 ms 2023-11-02 03:53:37,655 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:37,655 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:37,655 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for c21eb15d0957cd8a2c44a1d684d8b7de: 2023-11-02 03:53:37,658 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-11-02 03:53:37,659 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1698897217658"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897217658"}]},"ts":"1698897217658"} 2023-11-02 03:53:37,661 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-02 03:53:37,662 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-02 03:53:37,662 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897217662"}]},"ts":"1698897217662"} 2023-11-02 03:53:37,663 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-11-02 03:53:37,666 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=c21eb15d0957cd8a2c44a1d684d8b7de, ASSIGN}] 2023-11-02 03:53:37,668 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=c21eb15d0957cd8a2c44a1d684d8b7de, ASSIGN 2023-11-02 03:53:37,669 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=c21eb15d0957cd8a2c44a1d684d8b7de, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=false 2023-11-02 03:53:37,820 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=c21eb15d0957cd8a2c44a1d684d8b7de, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:37,820 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1698897217820"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897217820"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897217820"}]},"ts":"1698897217820"} 2023-11-02 03:53:37,822 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure c21eb15d0957cd8a2c44a1d684d8b7de, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:53:37,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-02 03:53:37,974 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:37,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:37,978 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c21eb15d0957cd8a2c44a1d684d8b7de, NAME => 'testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de.', STARTKEY => '', ENDKEY => ''} 2023-11-02 03:53:37,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy c21eb15d0957cd8a2c44a1d684d8b7de 2023-11-02 03:53:37,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:53:37,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c21eb15d0957cd8a2c44a1d684d8b7de 2023-11-02 03:53:37,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c21eb15d0957cd8a2c44a1d684d8b7de 2023-11-02 03:53:37,980 INFO [StoreOpener-c21eb15d0957cd8a2c44a1d684d8b7de-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c21eb15d0957cd8a2c44a1d684d8b7de 2023-11-02 03:53:37,982 DEBUG [StoreOpener-c21eb15d0957cd8a2c44a1d684d8b7de-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1 2023-11-02 03:53:37,982 DEBUG [StoreOpener-c21eb15d0957cd8a2c44a1d684d8b7de-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1 2023-11-02 03:53:37,982 INFO [StoreOpener-c21eb15d0957cd8a2c44a1d684d8b7de-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c21eb15d0957cd8a2c44a1d684d8b7de columnFamilyName family1 2023-11-02 03:53:37,984 INFO [StoreOpener-c21eb15d0957cd8a2c44a1d684d8b7de-1] regionserver.HStore(310): Store=c21eb15d0957cd8a2c44a1d684d8b7de/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:53:37,985 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de 2023-11-02 03:53:37,985 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de 2023-11-02 03:53:37,988 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c21eb15d0957cd8a2c44a1d684d8b7de 2023-11-02 03:53:37,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:53:37,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c21eb15d0957cd8a2c44a1d684d8b7de; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74943519, jitterRate=0.11674545705318451}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-02 03:53:37,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c21eb15d0957cd8a2c44a1d684d8b7de: 2023-11-02 03:53:37,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de., pid=21, masterSystemTime=1698897217974 2023-11-02 03:53:37,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:37,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:37,994 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=c21eb15d0957cd8a2c44a1d684d8b7de, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:37,994 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1698897217993"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897217993"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897217993"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897217993"}]},"ts":"1698897217993"} 2023-11-02 03:53:37,997 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=21, resume processing ppid=20 2023-11-02 03:53:37,997 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure c21eb15d0957cd8a2c44a1d684d8b7de, server=jenkins-hbase4.apache.org,36627,1698897201574 in 173 msec 2023-11-02 03:53:37,999 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=20, resume processing ppid=19 2023-11-02 03:53:37,999 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=c21eb15d0957cd8a2c44a1d684d8b7de, ASSIGN in 331 msec 2023-11-02 03:53:38,000 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-02 03:53:38,000 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897218000"}]},"ts":"1698897218000"} 2023-11-02 03:53:38,001 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-11-02 03:53:38,003 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-11-02 03:53:38,005 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 1.1840 sec 2023-11-02 03:53:38,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-11-02 03:53:38,931 INFO [Listener at localhost/43647] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-11-02 03:53:38,967 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:38,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:38,969 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-02 03:53:38,969 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:38,970 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-02 03:53:38,970 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-02 03:53:38,970 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:38,970 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:38,972 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:38,972 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:38,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:38,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:38,972 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:38,972 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:38,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:38,973 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:38,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:38,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:38,973 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:38,974 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:38,974 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-02 03:53:38,974 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:38,974 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-02 03:53:38,975 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:38,975 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:38,975 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:38,975 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. started... 2023-11-02 03:53:38,975 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c21eb15d0957cd8a2c44a1d684d8b7de 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-02 03:53:39,004 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130a891e0d5d5524168a4650209d2df0a53 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130a891e0d5d5524168a4650209d2df0a53 2023-11-02 03:53:39,008 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/ccaa6c8ff363489c94c8d0ea6da66b9e 2023-11-02 03:53:39,014 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/ccaa6c8ff363489c94c8d0ea6da66b9e as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/ccaa6c8ff363489c94c8d0ea6da66b9e 2023-11-02 03:53:39,020 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/ccaa6c8ff363489c94c8d0ea6da66b9e, entries=1000, sequenceid=5, filesize=166.0 K 2023-11-02 03:53:39,021 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for c21eb15d0957cd8a2c44a1d684d8b7de in 46ms, sequenceid=5, compaction requested=false 2023-11-02 03:53:39,021 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-11-02 03:53:39,022 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c21eb15d0957cd8a2c44a1d684d8b7de: 2023-11-02 03:53:39,022 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:39,022 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:39,023 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:39,023 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,023 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-02 03:53:39,023 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-02 03:53:39,024 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,024 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:39,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:39,025 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,025 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:39,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:39,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:39,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:39,026 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-02 03:53:39,026 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@45e62e53[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:39,026 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-02 03:53:39,026 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,028 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,029 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-02 03:53:39,029 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-02 03:53:39,029 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:39,029 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,029 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:39,030 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-02 03:53:39,030 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,030 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:39,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:39,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:39,030 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-02 03:53:39,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:39,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:39,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:39,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:39,034 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-02 03:53:39,034 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:39,034 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:39,034 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:53:39,035 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,035 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:39,036 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,036 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:39,036 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:39,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:39,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:39,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:39,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:39,037 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:39,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,042 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:39,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,054 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,054 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:39,054 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,055 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:39,055 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-02 03:53:39,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:39,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:39,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:39,055 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:39,055 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,056 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,056 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,056 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:39,056 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,056 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:39,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:39,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:39,156 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:39,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:39,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:39,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:39,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,166 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-02 03:53:39,166 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:39,166 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-02 03:53:39,167 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-02 03:53:39,167 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,167 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,168 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,168 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:39,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:39,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:39,169 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,169 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:39,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,170 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:39,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,170 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,170 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-02 03:53:39,170 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,170 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-02 03:53:39,170 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:39,171 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-02 03:53:39,171 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:39,171 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:39,171 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:39,171 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. started... 2023-11-02 03:53:39,171 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c21eb15d0957cd8a2c44a1d684d8b7de 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-02 03:53:39,592 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128df00cff0e8504be88302dbd6ff98c7e1 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128df00cff0e8504be88302dbd6ff98c7e1 2023-11-02 03:53:39,601 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/66d1b99a4aca425d82c327a3ebcf2d5e 2023-11-02 03:53:39,608 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/66d1b99a4aca425d82c327a3ebcf2d5e as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/66d1b99a4aca425d82c327a3ebcf2d5e 2023-11-02 03:53:39,618 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/66d1b99a4aca425d82c327a3ebcf2d5e, entries=1, sequenceid=9, filesize=5.0 K 2023-11-02 03:53:39,619 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for c21eb15d0957cd8a2c44a1d684d8b7de in 448ms, sequenceid=9, compaction requested=false 2023-11-02 03:53:39,619 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c21eb15d0957cd8a2c44a1d684d8b7de: 2023-11-02 03:53:39,620 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:39,620 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:39,620 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:39,620 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,620 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-02 03:53:39,620 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-02 03:53:39,622 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,622 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:39,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:39,622 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,622 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:39,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:39,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:39,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:39,624 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-02 03:53:39,624 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@13420af7[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:39,624 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-02 03:53:39,624 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,626 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,626 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-02 03:53:39,626 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-02 03:53:39,626 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:39,626 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,626 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:39,628 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,628 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-02 03:53:39,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:39,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:39,629 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:39,629 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-02 03:53:39,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:39,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:39,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:39,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:39,632 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:53:39,632 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-02 03:53:39,632 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:39,632 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:39,632 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,632 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:39,634 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,634 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:39,634 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:39,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:39,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:39,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:39,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:39,635 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:39,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,638 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:39,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,640 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,641 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,641 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,641 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,641 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,641 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,641 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,641 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:39,641 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:39,641 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-02 03:53:39,642 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:39,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:39,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:39,641 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:39,642 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:39,642 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:39,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:39,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:39,742 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:39,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:39,749 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:39,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:39,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,751 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-02 03:53:39,751 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:39,751 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-02 03:53:39,751 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-02 03:53:39,752 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,752 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,753 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,753 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:39,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:39,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:39,754 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:39,754 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:39,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:39,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,755 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,755 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-02 03:53:39,755 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:39,755 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-02 03:53:39,755 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:39,756 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-02 03:53:39,756 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:39,756 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:39,756 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:39,756 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. started... 2023-11-02 03:53:39,756 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c21eb15d0957cd8a2c44a1d684d8b7de 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:40,171 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201c7998ae57e3c41dd9d9ee756ee6b8cbc to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201c7998ae57e3c41dd9d9ee756ee6b8cbc 2023-11-02 03:53:40,577 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/2af2d336b105472981ab805c55c9360a 2023-11-02 03:53:40,586 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/2af2d336b105472981ab805c55c9360a as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/2af2d336b105472981ab805c55c9360a 2023-11-02 03:53:40,592 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/2af2d336b105472981ab805c55c9360a, entries=1, sequenceid=13, filesize=4.9 K 2023-11-02 03:53:40,593 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c21eb15d0957cd8a2c44a1d684d8b7de in 837ms, sequenceid=13, compaction requested=true 2023-11-02 03:53:40,593 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c21eb15d0957cd8a2c44a1d684d8b7de: 2023-11-02 03:53:40,593 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:40,594 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:40,594 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:40,594 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:40,594 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-02 03:53:40,594 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-02 03:53:40,596 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:40,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:40,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:40,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:40,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:40,596 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:40,597 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,597 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:40,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:40,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:40,598 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-02 03:53:40,598 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@70c28071[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:40,598 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-02 03:53:40,598 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,600 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,600 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-02 03:53:40,600 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-02 03:53:40,600 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:40,600 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:40,600 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:40,602 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-02 03:53:40,602 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:40,603 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:40,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:40,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:40,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:40,603 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-02 03:53:40,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:40,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:40,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:40,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:40,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:40,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:40,607 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:53:40,607 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-02 03:53:40,608 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:40,608 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:40,608 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,608 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:40,610 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,610 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:40,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,610 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:40,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:40,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:40,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:40,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:40,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,611 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:40,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:40,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:40,613 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:40,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:40,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:40,615 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:40,616 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:40,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:40,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:40,621 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:40,621 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,621 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,621 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:40,621 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,621 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,621 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,629 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:40,629 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-02 03:53:40,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:40,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-02 03:53:40,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:40,630 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:40,630 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:40,631 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:40,731 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:40,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:40,737 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:40,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:40,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,739 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-02 03:53:40,739 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:40,740 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-02 03:53:40,740 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-02 03:53:40,740 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,740 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,741 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:40,742 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:40,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:40,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:40,742 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:40,742 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:40,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,743 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:40,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,743 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,743 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-02 03:53:40,743 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:40,743 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-02 03:53:40,743 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:40,744 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-02 03:53:40,744 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:40,744 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:40,744 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:40,744 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. started... 2023-11-02 03:53:40,744 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c21eb15d0957cd8a2c44a1d684d8b7de 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:41,164 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512051a2189facdf54621ace503a700da432f to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512051a2189facdf54621ace503a700da432f 2023-11-02 03:53:41,177 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/b4de9773428a4c6a8290cede0de01473 2023-11-02 03:53:41,187 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/b4de9773428a4c6a8290cede0de01473 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/b4de9773428a4c6a8290cede0de01473 2023-11-02 03:53:41,194 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/b4de9773428a4c6a8290cede0de01473, entries=1, sequenceid=17, filesize=4.9 K 2023-11-02 03:53:41,195 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c21eb15d0957cd8a2c44a1d684d8b7de in 451ms, sequenceid=17, compaction requested=true 2023-11-02 03:53:41,196 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c21eb15d0957cd8a2c44a1d684d8b7de: 2023-11-02 03:53:41,196 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:41,196 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:41,196 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:41,196 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,196 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-02 03:53:41,196 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-02 03:53:41,199 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,199 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:41,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:41,199 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,200 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:41,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:41,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:41,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:41,202 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-02 03:53:41,202 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f6d11b5[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:41,202 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-02 03:53:41,202 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,204 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,204 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-02 03:53:41,204 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-02 03:53:41,204 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:41,204 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,205 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:41,210 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,210 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-02 03:53:41,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:41,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:41,210 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:41,210 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-02 03:53:41,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:41,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:41,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:41,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:41,215 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:53:41,215 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-02 03:53:41,215 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:41,215 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:41,216 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,216 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:41,218 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,218 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:41,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,218 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:41,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:41,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:41,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:41,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:41,219 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:41,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,221 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:41,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,223 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-02 03:53:41,226 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:41,227 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:41,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:41,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:41,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:41,227 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:41,227 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-02 03:53:41,227 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:41,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:41,227 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,228 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,228 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,228 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,228 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,228 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,228 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,228 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:41,228 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:41,328 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:41,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:41,335 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:41,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:41,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,338 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-02 03:53:41,338 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:41,338 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-02 03:53:41,338 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-02 03:53:41,339 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,339 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,340 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:41,340 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:41,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:41,341 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,341 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:41,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,341 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:41,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,342 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,342 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-02 03:53:41,342 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,342 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-02 03:53:41,343 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:41,343 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-02 03:53:41,343 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:41,343 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:41,343 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:41,343 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. started... 2023-11-02 03:53:41,343 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c21eb15d0957cd8a2c44a1d684d8b7de 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:41,361 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228ebcaa0696fec44d9be71a37a1662b259 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228ebcaa0696fec44d9be71a37a1662b259 2023-11-02 03:53:41,767 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/9bf43ca9c87b41c18eca95b3ba0b2910 2023-11-02 03:53:41,775 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/9bf43ca9c87b41c18eca95b3ba0b2910 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/9bf43ca9c87b41c18eca95b3ba0b2910 2023-11-02 03:53:41,782 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/9bf43ca9c87b41c18eca95b3ba0b2910, entries=1, sequenceid=21, filesize=4.9 K 2023-11-02 03:53:41,784 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c21eb15d0957cd8a2c44a1d684d8b7de in 441ms, sequenceid=21, compaction requested=true 2023-11-02 03:53:41,784 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c21eb15d0957cd8a2c44a1d684d8b7de: 2023-11-02 03:53:41,784 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:41,784 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:41,784 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:41,784 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,784 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-02 03:53:41,784 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-02 03:53:41,787 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:41,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:41,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:41,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:41,789 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,789 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,789 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:41,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:41,790 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-02 03:53:41,790 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b0d36bd[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:41,790 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-02 03:53:41,790 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,792 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,792 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-02 03:53:41,792 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,792 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:41,792 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-02 03:53:41,792 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:41,794 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-02 03:53:41,794 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:41,794 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,794 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-02 03:53:41,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:41,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:41,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:41,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:41,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:41,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:41,798 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:53:41,798 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-02 03:53:41,798 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:41,798 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:41,798 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,798 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:41,800 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,800 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:41,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,800 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:41,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:41,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:41,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:41,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,802 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:41,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:41,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:41,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,808 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,812 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,812 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,812 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,812 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,812 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,812 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,812 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,812 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:41,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-02 03:53:41,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:41,812 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:41,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:41,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:41,812 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:41,813 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:41,813 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:41,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:41,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:41,913 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:41,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:41,920 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:41,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:41,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,922 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-02 03:53:41,922 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:41,922 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-02 03:53:41,923 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-02 03:53:41,923 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,923 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,925 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:41,925 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:41,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:41,925 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:41,926 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:41,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,926 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:41,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,927 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,927 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-02 03:53:41,927 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:41,927 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-02 03:53:41,927 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:41,927 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-02 03:53:41,927 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:41,928 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:41,928 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:41,928 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. started... 2023-11-02 03:53:41,928 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c21eb15d0957cd8a2c44a1d684d8b7de 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:42,347 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512312adf5ce55cf640d1a4c87cc8e5e1bbf6 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512312adf5ce55cf640d1a4c87cc8e5e1bbf6 2023-11-02 03:53:42,753 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/fe5ff7be0a074ecbb4cd7229b7df4223 2023-11-02 03:53:42,760 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/fe5ff7be0a074ecbb4cd7229b7df4223 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/fe5ff7be0a074ecbb4cd7229b7df4223 2023-11-02 03:53:42,766 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/fe5ff7be0a074ecbb4cd7229b7df4223, entries=1, sequenceid=25, filesize=4.9 K 2023-11-02 03:53:42,768 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c21eb15d0957cd8a2c44a1d684d8b7de in 840ms, sequenceid=25, compaction requested=true 2023-11-02 03:53:42,768 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c21eb15d0957cd8a2c44a1d684d8b7de: 2023-11-02 03:53:42,768 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:42,768 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:42,768 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:42,768 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:42,768 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-02 03:53:42,768 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-02 03:53:42,855 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:42,855 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:42,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:42,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:42,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:42,855 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:42,855 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:42,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:42,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:42,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:42,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:42,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:42,857 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-02 03:53:42,857 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@581b8a99[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:42,857 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-02 03:53:42,857 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,047 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,047 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-02 03:53:43,048 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:43,048 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:43,048 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-02 03:53:43,048 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:43,155 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-02 03:53:43,155 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:43,155 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:43,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:43,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:43,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:43,155 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-02 03:53:43,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:43,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:43,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:43,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:43,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:43,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:43,159 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:53:43,159 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-02 03:53:43,159 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:43,159 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:43,159 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,159 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:43,232 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,232 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,232 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:43,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:43,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:43,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:43,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:43,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:43,233 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:43,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:43,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:43,234 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:43,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:43,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:43,324 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:43,324 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:43,324 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,324 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:43,324 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,324 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-02 03:53:43,325 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:43,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:43,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:43,324 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:43,324 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:43,325 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,325 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,325 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,325 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:43,326 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:43,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:43,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:43,426 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:43,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:43,431 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:43,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:43,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,434 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-02 03:53:43,434 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:43,435 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-02 03:53:43,435 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-02 03:53:43,437 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,437 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,478 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:43,478 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:43,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:43,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:43,478 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:43,479 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:43,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,479 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:43,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,479 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,480 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-02 03:53:43,480 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:43,480 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-02 03:53:43,480 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:43,480 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-02 03:53:43,481 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:43,481 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:43,481 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:43,481 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. started... 2023-11-02 03:53:43,481 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c21eb15d0957cd8a2c44a1d684d8b7de 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:43,896 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010154e0115ec5384495b8387701378587a4 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010154e0115ec5384495b8387701378587a4 2023-11-02 03:53:44,302 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/668545ca7d404b098db20eb7c6d604a1 2023-11-02 03:53:44,308 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/668545ca7d404b098db20eb7c6d604a1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/668545ca7d404b098db20eb7c6d604a1 2023-11-02 03:53:44,314 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/668545ca7d404b098db20eb7c6d604a1, entries=1, sequenceid=29, filesize=4.9 K 2023-11-02 03:53:44,315 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c21eb15d0957cd8a2c44a1d684d8b7de in 834ms, sequenceid=29, compaction requested=true 2023-11-02 03:53:44,315 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c21eb15d0957cd8a2c44a1d684d8b7de: 2023-11-02 03:53:44,315 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:44,315 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:44,315 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:44,315 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:44,315 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-02 03:53:44,315 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-02 03:53:44,440 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:44,440 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:44,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:44,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:44,440 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,440 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:44,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:44,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:44,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:44,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:44,442 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-02 03:53:44,442 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3c3e58e3[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:44,442 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-02 03:53:44,442 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,516 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,517 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-02 03:53:44,517 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-02 03:53:44,517 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:44,517 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:44,517 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:44,562 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:44,562 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-02 03:53:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:44,562 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:44,563 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-02 03:53:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:44,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:44,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:44,566 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:53:44,566 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-02 03:53:44,566 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:44,566 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:44,566 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,566 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:44,616 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,616 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:44,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:44,617 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:44,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:44,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:44,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:44,617 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:44,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:44,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:44,619 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:44,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:44,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:44,670 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:44,670 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:44,670 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:44,670 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-02 03:53:44,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:44,670 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:44,670 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:44,670 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,670 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:44,671 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:44,671 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,671 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,671 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,671 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:44,671 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:44,671 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:44,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:44,772 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:44,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:44,780 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:44,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:44,782 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,782 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-02 03:53:44,782 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:44,782 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-02 03:53:44,782 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-02 03:53:44,783 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,783 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,785 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:44,785 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:44,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:44,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:44,786 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:44,786 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:44,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,786 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:44,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,787 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,787 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-02 03:53:44,787 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:44,787 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-02 03:53:44,787 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:44,787 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-02 03:53:44,788 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:44,788 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:44,788 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:44,788 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. started... 2023-11-02 03:53:44,788 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c21eb15d0957cd8a2c44a1d684d8b7de 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:45,204 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103a88588e149f44525a8f159a34d08f2ab to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103a88588e149f44525a8f159a34d08f2ab 2023-11-02 03:53:45,211 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/4831e3e66cd447299dbebd6a261a5dec 2023-11-02 03:53:45,217 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/4831e3e66cd447299dbebd6a261a5dec as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/4831e3e66cd447299dbebd6a261a5dec 2023-11-02 03:53:45,222 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/4831e3e66cd447299dbebd6a261a5dec, entries=1, sequenceid=33, filesize=4.9 K 2023-11-02 03:53:45,223 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c21eb15d0957cd8a2c44a1d684d8b7de in 435ms, sequenceid=33, compaction requested=true 2023-11-02 03:53:45,224 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c21eb15d0957cd8a2c44a1d684d8b7de: 2023-11-02 03:53:45,224 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:45,224 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:45,224 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:45,224 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,224 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-02 03:53:45,224 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-02 03:53:45,226 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,226 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:45,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:45,227 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,227 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:45,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:45,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:45,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:45,228 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-02 03:53:45,228 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b990668[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:45,228 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-02 03:53:45,228 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,231 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,231 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-02 03:53:45,231 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-02 03:53:45,231 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:45,231 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,231 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:45,233 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-02 03:53:45,233 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:45,233 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-02 03:53:45,234 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:45,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:45,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:45,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:45,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:45,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:45,238 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:53:45,238 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-02 03:53:45,238 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:45,238 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:45,238 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,238 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:45,240 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,240 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:45,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:45,240 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:45,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:45,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:45,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:45,241 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:45,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,243 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:45,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,246 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:45,246 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,246 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:45,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:45,246 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,246 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,246 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,246 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,246 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,246 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,246 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:45,246 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-02 03:53:45,247 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:45,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:45,247 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:45,247 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:45,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:45,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:45,347 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:45,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:45,354 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:45,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:45,357 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,357 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-02 03:53:45,357 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:45,358 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-02 03:53:45,358 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-02 03:53:45,358 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,358 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,359 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:45,359 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:45,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:45,360 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,360 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:45,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,360 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:45,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,361 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,361 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-02 03:53:45,361 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,361 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-02 03:53:45,361 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:45,361 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-02 03:53:45,362 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:45,362 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:45,362 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:45,362 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. started... 2023-11-02 03:53:45,362 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c21eb15d0957cd8a2c44a1d684d8b7de 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:45,376 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509074c34f63315ef47768aa218551826f9b3 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201509074c34f63315ef47768aa218551826f9b3 2023-11-02 03:53:45,784 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/723a488f0bce420fb9c999401ba4ab88 2023-11-02 03:53:45,790 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/723a488f0bce420fb9c999401ba4ab88 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/723a488f0bce420fb9c999401ba4ab88 2023-11-02 03:53:45,795 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/723a488f0bce420fb9c999401ba4ab88, entries=1, sequenceid=37, filesize=4.9 K 2023-11-02 03:53:45,796 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c21eb15d0957cd8a2c44a1d684d8b7de in 434ms, sequenceid=37, compaction requested=true 2023-11-02 03:53:45,796 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c21eb15d0957cd8a2c44a1d684d8b7de: 2023-11-02 03:53:45,796 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:45,796 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:45,796 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:45,796 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,796 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-02 03:53:45,796 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-02 03:53:45,798 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,798 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:45,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:45,799 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:45,799 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:45,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:45,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:45,800 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-02 03:53:45,800 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@49936f78[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:45,800 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-02 03:53:45,800 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,802 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,802 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-02 03:53:45,802 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-02 03:53:45,802 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:45,802 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,802 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:45,804 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-02 03:53:45,804 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,804 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:45,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:45,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:45,804 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-02 03:53:45,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:45,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:45,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:45,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:45,807 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:53:45,807 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-02 03:53:45,807 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:45,807 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:45,807 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,807 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:45,809 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,809 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:45,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:45,809 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:45,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:45,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:45,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:45,810 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:45,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,811 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:45,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,817 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,817 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,817 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:45,817 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:45,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:45,817 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:45,817 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-02 03:53:45,817 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:45,817 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,817 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:45,818 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:45,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:45,818 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,818 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:45,818 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:45,818 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,919 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:45,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:45,924 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:45,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:45,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,926 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-11-02 03:53:45,926 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:45,926 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-02 03:53:45,926 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-02 03:53:45,927 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,927 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,928 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,928 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:45,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:45,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:45,929 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:45,929 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:45,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,929 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:45,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,930 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,930 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-11-02 03:53:45,930 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:45,930 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-02 03:53:45,930 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:45,931 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-02 03:53:45,932 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:45,932 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:45,932 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:45,932 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. started... 2023-11-02 03:53:45,932 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c21eb15d0957cd8a2c44a1d684d8b7de 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:46,348 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120ece0b35507b84c72b399af646bdcdef8 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120ece0b35507b84c72b399af646bdcdef8 2023-11-02 03:53:46,754 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/fe00f37463ec40aea57eecbc4506725a 2023-11-02 03:53:46,761 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/fe00f37463ec40aea57eecbc4506725a as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/fe00f37463ec40aea57eecbc4506725a 2023-11-02 03:53:46,766 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/fe00f37463ec40aea57eecbc4506725a, entries=1, sequenceid=41, filesize=4.9 K 2023-11-02 03:53:46,767 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c21eb15d0957cd8a2c44a1d684d8b7de in 835ms, sequenceid=41, compaction requested=true 2023-11-02 03:53:46,767 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c21eb15d0957cd8a2c44a1d684d8b7de: 2023-11-02 03:53:46,767 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:46,767 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:46,767 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:46,767 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:46,767 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-11-02 03:53:46,767 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-11-02 03:53:46,769 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:46,769 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:46,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:46,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:46,770 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,770 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:46,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:46,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:46,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:46,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:46,771 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-11-02 03:53:46,771 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4dd87b91[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:46,771 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-02 03:53:46,771 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,773 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,773 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-02 03:53:46,773 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-11-02 03:53:46,773 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:46,774 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:46,774 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:46,775 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:46,775 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-02 03:53:46,775 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:46,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:46,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:46,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:46,775 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-11-02 03:53:46,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:46,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:46,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:46,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:46,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:46,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:46,778 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:53:46,778 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-11-02 03:53:46,778 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:46,778 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:46,778 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,778 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:46,780 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,780 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,780 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:46,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:46,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:46,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:46,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:46,780 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:46,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:46,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:46,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,781 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:46,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:46,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:46,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:46,784 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:46,784 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,784 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,784 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:46,784 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,784 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,784 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,784 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:46,784 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-02 03:53:46,784 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:46,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-11-02 03:53:46,785 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:46,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:46,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:46,785 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:46,785 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:46,786 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:46,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:46,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:46,886 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:46,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:46,889 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41275] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-11-02 03:53:46,890 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] procedure2.ProcedureExecutor(1028): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-02 03:53:46,891 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-02 03:53:46,892 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:53:46,892 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698897826892 2023-11-02 03:53:46,893 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-02 03:53:46,894 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-11-02 03:53:46,895 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-02 03:53:46,896 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] zookeeper.ReadOnlyZKClient(139): Connect 0x685d2806 to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:53:46,902 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@41d67fea, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:53:46,903 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-02 03:53:47,365 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601036bae5c0bbfe849258a8267976aa533b2 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601036bae5c0bbfe849258a8267976aa533b2 2023-11-02 03:53:47,366 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-02 03:53:47,379 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-02 03:53:47,380 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51534, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-02 03:53:47,387 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-02 03:53:47,388 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52114, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-02 03:53:47,397 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/2fa567bd7e704ae48024b690a697be78 first=Optional[r5] last=Optional[r6] 2023-11-02 03:53:47,398 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/2fa567bd7e704ae48024b690a697be78}] 2023-11-02 03:53:47,402 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/2fa567bd7e704ae48024b690a697be78 for inclusion in c21eb15d0957cd8a2c44a1d684d8b7de/family1 2023-11-02 03:53:47,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-11-02 03:53:47,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last= 2023-11-02 03:53:47,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for c21eb15d0957cd8a2c44a1d684d8b7de: 2023-11-02 03:53:47,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/2fa567bd7e704ae48024b690a697be78 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__pn42uv36nqmtsqhhscpg4evr6v6266cakpgcshu1u4qbbsiu2jvhp6pg6tg0t6bj/family1/2fa567bd7e704ae48024b690a697be78 2023-11-02 03:53:47,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__pn42uv36nqmtsqhhscpg4evr6v6266cakpgcshu1u4qbbsiu2jvhp6pg6tg0t6bj/family1/2fa567bd7e704ae48024b690a697be78 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/be6fe92d5ac64a5ea5cc0437209e5a39_SeqId_44_ 2023-11-02 03:53:47,408 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__pn42uv36nqmtsqhhscpg4evr6v6266cakpgcshu1u4qbbsiu2jvhp6pg6tg0t6bj/family1/2fa567bd7e704ae48024b690a697be78 into c21eb15d0957cd8a2c44a1d684d8b7de/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/be6fe92d5ac64a5ea5cc0437209e5a39_SeqId_44_ - updating store file list. 2023-11-02 03:53:47,413 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/be6fe92d5ac64a5ea5cc0437209e5a39_SeqId_44_ into c21eb15d0957cd8a2c44a1d684d8b7de/family1 2023-11-02 03:53:47,413 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__pn42uv36nqmtsqhhscpg4evr6v6266cakpgcshu1u4qbbsiu2jvhp6pg6tg0t6bj/family1/2fa567bd7e704ae48024b690a697be78 into c21eb15d0957cd8a2c44a1d684d8b7de/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/be6fe92d5ac64a5ea5cc0437209e5a39_SeqId_44_) 2023-11-02 03:53:47,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__pn42uv36nqmtsqhhscpg4evr6v6266cakpgcshu1u4qbbsiu2jvhp6pg6tg0t6bj/family1/2fa567bd7e704ae48024b690a697be78 2023-11-02 03:53:47,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-02 03:53:47,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-02 03:53:47,417 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(1912): c21eb15d0957cd8a2c44a1d684d8b7de/family1 is initiating minor compaction 2023-11-02 03:53:47,418 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@55deeb74; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:53:47,418 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c21eb15d0957cd8a2c44a1d684d8b7de/family1 in testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:47,418 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/66d1b99a4aca425d82c327a3ebcf2d5e, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/2af2d336b105472981ab805c55c9360a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/b4de9773428a4c6a8290cede0de01473, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/9bf43ca9c87b41c18eca95b3ba0b2910, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/fe5ff7be0a074ecbb4cd7229b7df4223, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/668545ca7d404b098db20eb7c6d604a1, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/4831e3e66cd447299dbebd6a261a5dec, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/723a488f0bce420fb9c999401ba4ab88, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/fe00f37463ec40aea57eecbc4506725a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/be6fe92d5ac64a5ea5cc0437209e5a39_SeqId_44_] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp, totalSize=49.7 K 2023-11-02 03:53:47,418 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 66d1b99a4aca425d82c327a3ebcf2d5e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-02 03:53:47,418 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 2af2d336b105472981ab805c55c9360a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-02 03:53:47,418 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting b4de9773428a4c6a8290cede0de01473, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-02 03:53:47,418 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 9bf43ca9c87b41c18eca95b3ba0b2910, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-02 03:53:47,418 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting fe5ff7be0a074ecbb4cd7229b7df4223, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-02 03:53:47,418 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 668545ca7d404b098db20eb7c6d604a1, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-02 03:53:47,418 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 4831e3e66cd447299dbebd6a261a5dec, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-02 03:53:47,419 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 723a488f0bce420fb9c999401ba4ab88, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-02 03:53:47,419 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting fe00f37463ec40aea57eecbc4506725a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-02 03:53:47,419 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting be6fe92d5ac64a5ea5cc0437209e5a39_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-02 03:53:47,420 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-02 03:53:47,421 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:53:47,423 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010154e0115ec5384495b8387701378587a4 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010154e0115ec5384495b8387701378587a4 2023-11-02 03:53:47,424 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103a88588e149f44525a8f159a34d08f2ab to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103a88588e149f44525a8f159a34d08f2ab 2023-11-02 03:53:47,424 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-11-02 03:53:47,424 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-11-02 03:53:47,425 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-02 03:53:47,425 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-02 03:53:47,458 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] throttle.PressureAwareThroughputController(145): c21eb15d0957cd8a2c44a1d684d8b7de#family1#compaction#22 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:53:47,882 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/5300de6b71054ceb846ccc458d23f5dc as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/5300de6b71054ceb846ccc458d23f5dc 2023-11-02 03:53:47,891 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in c21eb15d0957cd8a2c44a1d684d8b7de/family1 of c21eb15d0957cd8a2c44a1d684d8b7de into 5300de6b71054ceb846ccc458d23f5dc(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-02 03:53:47,891 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c21eb15d0957cd8a2c44a1d684d8b7de: 2023-11-02 03:53:47,891 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1698897227417; duration=0sec 2023-11-02 03:53:47,891 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:53:47,895 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231aa168c2576624381842698b6ed236e17 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231aa168c2576624381842698b6ed236e17 2023-11-02 03:53:47,896 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-02 03:53:47,944 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/e00dc99017bf4639b9c2dd872516c6fa first=Optional[r1] last=Optional[r4] 2023-11-02 03:53:47,945 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/e00dc99017bf4639b9c2dd872516c6fa}] 2023-11-02 03:53:47,949 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/e00dc99017bf4639b9c2dd872516c6fa for inclusion in c21eb15d0957cd8a2c44a1d684d8b7de/family1 2023-11-02 03:53:47,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-11-02 03:53:47,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last= 2023-11-02 03:53:47,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for c21eb15d0957cd8a2c44a1d684d8b7de: 2023-11-02 03:53:47,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/e00dc99017bf4639b9c2dd872516c6fa to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__gj177bi0jds82agqplagf4gfnf8iq7ii3e960on7t800kcc42hh29lb9u4iv2bce/family1/e00dc99017bf4639b9c2dd872516c6fa 2023-11-02 03:53:47,968 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/66d1b99a4aca425d82c327a3ebcf2d5e, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/2af2d336b105472981ab805c55c9360a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/b4de9773428a4c6a8290cede0de01473, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/9bf43ca9c87b41c18eca95b3ba0b2910, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/fe5ff7be0a074ecbb4cd7229b7df4223, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/668545ca7d404b098db20eb7c6d604a1, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/4831e3e66cd447299dbebd6a261a5dec, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/723a488f0bce420fb9c999401ba4ab88, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/fe00f37463ec40aea57eecbc4506725a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/be6fe92d5ac64a5ea5cc0437209e5a39_SeqId_44_] to archive 2023-11-02 03:53:47,969 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:53:47,972 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/66d1b99a4aca425d82c327a3ebcf2d5e to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/66d1b99a4aca425d82c327a3ebcf2d5e 2023-11-02 03:53:47,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__gj177bi0jds82agqplagf4gfnf8iq7ii3e960on7t800kcc42hh29lb9u4iv2bce/family1/e00dc99017bf4639b9c2dd872516c6fa as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/6ef96fbb6fc3421188c7a20c6461b010_SeqId_47_ 2023-11-02 03:53:47,973 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__gj177bi0jds82agqplagf4gfnf8iq7ii3e960on7t800kcc42hh29lb9u4iv2bce/family1/e00dc99017bf4639b9c2dd872516c6fa into c21eb15d0957cd8a2c44a1d684d8b7de/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/6ef96fbb6fc3421188c7a20c6461b010_SeqId_47_ - updating store file list. 2023-11-02 03:53:47,974 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/2af2d336b105472981ab805c55c9360a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/2af2d336b105472981ab805c55c9360a 2023-11-02 03:53:47,976 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/b4de9773428a4c6a8290cede0de01473 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/b4de9773428a4c6a8290cede0de01473 2023-11-02 03:53:47,979 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/9bf43ca9c87b41c18eca95b3ba0b2910 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/9bf43ca9c87b41c18eca95b3ba0b2910 2023-11-02 03:53:47,981 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/fe5ff7be0a074ecbb4cd7229b7df4223 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/fe5ff7be0a074ecbb4cd7229b7df4223 2023-11-02 03:53:47,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/6ef96fbb6fc3421188c7a20c6461b010_SeqId_47_ into c21eb15d0957cd8a2c44a1d684d8b7de/family1 2023-11-02 03:53:47,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__gj177bi0jds82agqplagf4gfnf8iq7ii3e960on7t800kcc42hh29lb9u4iv2bce/family1/e00dc99017bf4639b9c2dd872516c6fa into c21eb15d0957cd8a2c44a1d684d8b7de/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/6ef96fbb6fc3421188c7a20c6461b010_SeqId_47_) 2023-11-02 03:53:47,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__gj177bi0jds82agqplagf4gfnf8iq7ii3e960on7t800kcc42hh29lb9u4iv2bce/family1/e00dc99017bf4639b9c2dd872516c6fa 2023-11-02 03:53:47,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-02 03:53:47,983 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/668545ca7d404b098db20eb7c6d604a1 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/668545ca7d404b098db20eb7c6d604a1 2023-11-02 03:53:47,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-02 03:53:47,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(1912): c21eb15d0957cd8a2c44a1d684d8b7de/family1 is initiating minor compaction (all files) 2023-11-02 03:53:47,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@51711965; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:53:47,984 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c21eb15d0957cd8a2c44a1d684d8b7de/family1 in testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:53:47,985 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/ccaa6c8ff363489c94c8d0ea6da66b9e, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/5300de6b71054ceb846ccc458d23f5dc, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/6ef96fbb6fc3421188c7a20c6461b010_SeqId_47_] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp, totalSize=178.1 K 2023-11-02 03:53:47,987 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting ccaa6c8ff363489c94c8d0ea6da66b9e, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878824741 2023-11-02 03:53:47,988 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-02 03:53:47,988 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 5300de6b71054ceb846ccc458d23f5dc, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621224741 2023-11-02 03:53:47,989 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/4831e3e66cd447299dbebd6a261a5dec to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/4831e3e66cd447299dbebd6a261a5dec 2023-11-02 03:53:47,989 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:53:47,989 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 6ef96fbb6fc3421188c7a20c6461b010_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965224741 2023-11-02 03:53:47,991 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/723a488f0bce420fb9c999401ba4ab88 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/723a488f0bce420fb9c999401ba4ab88 2023-11-02 03:53:47,995 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201c7998ae57e3c41dd9d9ee756ee6b8cbc to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151201c7998ae57e3c41dd9d9ee756ee6b8cbc 2023-11-02 03:53:47,999 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512051a2189facdf54621ace503a700da432f to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512051a2189facdf54621ace503a700da432f 2023-11-02 03:53:48,000 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/fe00f37463ec40aea57eecbc4506725a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/fe00f37463ec40aea57eecbc4506725a 2023-11-02 03:53:48,002 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228ebcaa0696fec44d9be71a37a1662b259 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228ebcaa0696fec44d9be71a37a1662b259 2023-11-02 03:53:48,004 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/be6fe92d5ac64a5ea5cc0437209e5a39_SeqId_44_ to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/be6fe92d5ac64a5ea5cc0437209e5a39_SeqId_44_ 2023-11-02 03:53:48,005 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512312adf5ce55cf640d1a4c87cc8e5e1bbf6 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512312adf5ce55cf640d1a4c87cc8e5e1bbf6 2023-11-02 03:53:48,005 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-11-02 03:53:48,005 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-11-02 03:53:48,006 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-02 03:53:48,006 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-02 03:53:48,044 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] throttle.PressureAwareThroughputController(145): c21eb15d0957cd8a2c44a1d684d8b7de#family1#compaction#23 average throughput is 7.78 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:53:48,874 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130143f8301e9264c1a86a83617dd97d052 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130143f8301e9264c1a86a83617dd97d052 2023-11-02 03:53:48,875 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-02 03:53:48,901 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/7cdeafa5704c47c78adc60d581f6439d first=Optional[r00] last=Optional[r8] 2023-11-02 03:53:48,901 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/7cdeafa5704c47c78adc60d581f6439d}] 2023-11-02 03:53:48,904 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/7cdeafa5704c47c78adc60d581f6439d for inclusion in c21eb15d0957cd8a2c44a1d684d8b7de/family1 2023-11-02 03:53:48,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-11-02 03:53:48,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last= 2023-11-02 03:53:48,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for c21eb15d0957cd8a2c44a1d684d8b7de: 2023-11-02 03:53:48,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/7cdeafa5704c47c78adc60d581f6439d to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ht8ms2ka0v1fu2oqivfv5nhd6sbm9dmh3qajmi7b2vsjcvvndh74tt7bs4mq4jmu/family1/7cdeafa5704c47c78adc60d581f6439d 2023-11-02 03:53:48,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ht8ms2ka0v1fu2oqivfv5nhd6sbm9dmh3qajmi7b2vsjcvvndh74tt7bs4mq4jmu/family1/7cdeafa5704c47c78adc60d581f6439d as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/fef2db799d2d43aaaa1bf76a92045653_SeqId_49_ 2023-11-02 03:53:48,911 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ht8ms2ka0v1fu2oqivfv5nhd6sbm9dmh3qajmi7b2vsjcvvndh74tt7bs4mq4jmu/family1/7cdeafa5704c47c78adc60d581f6439d into c21eb15d0957cd8a2c44a1d684d8b7de/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/fef2db799d2d43aaaa1bf76a92045653_SeqId_49_ - updating store file list. 2023-11-02 03:53:48,917 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/fef2db799d2d43aaaa1bf76a92045653_SeqId_49_ into c21eb15d0957cd8a2c44a1d684d8b7de/family1 2023-11-02 03:53:48,917 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ht8ms2ka0v1fu2oqivfv5nhd6sbm9dmh3qajmi7b2vsjcvvndh74tt7bs4mq4jmu/family1/7cdeafa5704c47c78adc60d581f6439d into c21eb15d0957cd8a2c44a1d684d8b7de/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/fef2db799d2d43aaaa1bf76a92045653_SeqId_49_) 2023-11-02 03:53:48,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithMonthlyPolicy__ht8ms2ka0v1fu2oqivfv5nhd6sbm9dmh3qajmi7b2vsjcvvndh74tt7bs4mq4jmu/family1/7cdeafa5704c47c78adc60d581f6439d 2023-11-02 03:53:48,919 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-02 03:53:48,920 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:53:48,922 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120ece0b35507b84c72b399af646bdcdef8 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120ece0b35507b84c72b399af646bdcdef8 2023-11-02 03:53:48,924 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128df00cff0e8504be88302dbd6ff98c7e1 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128df00cff0e8504be88302dbd6ff98c7e1 2023-11-02 03:53:48,925 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130a891e0d5d5524168a4650209d2df0a53 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151130a891e0d5d5524168a4650209d2df0a53 2023-11-02 03:53:48,925 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-11-02 03:53:48,925 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-11-02 03:53:48,926 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-02 03:53:48,926 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-02 03:53:48,926 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-11-02 03:53:48,926 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-11-02 03:53:48,926 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:53:48,927 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-02 03:53:48,929 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.0380 sec 2023-11-02 03:53:49,032 INFO [Listener at localhost/43647] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=257 (was 252) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/cluster_a48e5c36-4405-e712-b078-c0f3440ea40f/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:45786 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:54798 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65185@0x685d2806-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:549) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:54606 [Waiting for operation #33] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c082619-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:54664 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c082619-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:45820 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:45744 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/93bb80fd7d8f4bd1953481c27bcd21ca java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:54526 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c082619-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c082619-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:45724 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c082619-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65185@0x685d2806-SendThread(127.0.0.1:65185) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:332) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1289) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/cluster_a48e5c36-4405-e712-b078-c0f3440ea40f/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:45740 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:45708 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c082619-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:45776 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6c082619-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:45896 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:65185@0x685d2806 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$88/1750819181.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_860291330_17 at /127.0.0.1:45696 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1105563298_17 at /127.0.0.1:45804 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=634 (was 636), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=344 (was 352), ProcessCount=170 (was 170), AvailableMemoryMB=11603 (was 11790) 2023-11-02 03:53:49,046 INFO [Listener at localhost/43647] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=257, OpenFileDescriptor=634, MaxFileDescriptor=60000, SystemLoadAverage=344, ProcessCount=170, AvailableMemoryMB=11602 2023-11-02 03:53:49,049 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-02 03:53:49,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:49,052 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-11-02 03:53:49,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-11-02 03:53:49,052 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-02 03:53:49,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-02 03:53:49,054 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d 2023-11-02 03:53:49,054 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d empty. 2023-11-02 03:53:49,055 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d 2023-11-02 03:53:49,055 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-11-02 03:53:49,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-02 03:53:49,291 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/.tmp/family1/93bb80fd7d8f4bd1953481c27bcd21ca as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/93bb80fd7d8f4bd1953481c27bcd21ca 2023-11-02 03:53:49,297 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in c21eb15d0957cd8a2c44a1d684d8b7de/family1 of c21eb15d0957cd8a2c44a1d684d8b7de into 93bb80fd7d8f4bd1953481c27bcd21ca(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-02 03:53:49,297 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c21eb15d0957cd8a2c44a1d684d8b7de: 2023-11-02 03:53:49,297 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1698897227984; duration=1sec 2023-11-02 03:53:49,297 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:53:49,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-02 03:53:49,357 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/ccaa6c8ff363489c94c8d0ea6da66b9e, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/5300de6b71054ceb846ccc458d23f5dc, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/6ef96fbb6fc3421188c7a20c6461b010_SeqId_47_] to archive 2023-11-02 03:53:49,358 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:53:49,360 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/ccaa6c8ff363489c94c8d0ea6da66b9e to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/ccaa6c8ff363489c94c8d0ea6da66b9e 2023-11-02 03:53:49,362 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/5300de6b71054ceb846ccc458d23f5dc to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/5300de6b71054ceb846ccc458d23f5dc 2023-11-02 03:53:49,364 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/6ef96fbb6fc3421188c7a20c6461b010_SeqId_47_ to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/family1/6ef96fbb6fc3421188c7a20c6461b010_SeqId_47_ 2023-11-02 03:53:49,464 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-11-02 03:53:49,466 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => d2c7ce048f92242cd7278c0667f09a6d, NAME => 'testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp 2023-11-02 03:53:49,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-02 03:53:49,872 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:53:49,872 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing d2c7ce048f92242cd7278c0667f09a6d, disabling compactions & flushes 2023-11-02 03:53:49,872 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:49,872 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:49,872 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. after waiting 0 ms 2023-11-02 03:53:49,872 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:49,872 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:49,872 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for d2c7ce048f92242cd7278c0667f09a6d: 2023-11-02 03:53:49,876 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-11-02 03:53:49,877 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1698897229877"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897229877"}]},"ts":"1698897229877"} 2023-11-02 03:53:49,879 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-02 03:53:49,880 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-02 03:53:49,880 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897229880"}]},"ts":"1698897229880"} 2023-11-02 03:53:49,881 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-11-02 03:53:49,895 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=d2c7ce048f92242cd7278c0667f09a6d, ASSIGN}] 2023-11-02 03:53:49,898 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=d2c7ce048f92242cd7278c0667f09a6d, ASSIGN 2023-11-02 03:53:49,899 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=d2c7ce048f92242cd7278c0667f09a6d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=false 2023-11-02 03:53:50,050 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=d2c7ce048f92242cd7278c0667f09a6d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:50,051 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1698897230050"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897230050"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897230050"}]},"ts":"1698897230050"} 2023-11-02 03:53:50,053 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure d2c7ce048f92242cd7278c0667f09a6d, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:53:50,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-02 03:53:50,205 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:50,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:50,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d2c7ce048f92242cd7278c0667f09a6d, NAME => 'testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d.', STARTKEY => '', ENDKEY => ''} 2023-11-02 03:53:50,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy d2c7ce048f92242cd7278c0667f09a6d 2023-11-02 03:53:50,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:53:50,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d2c7ce048f92242cd7278c0667f09a6d 2023-11-02 03:53:50,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d2c7ce048f92242cd7278c0667f09a6d 2023-11-02 03:53:50,210 INFO [StoreOpener-d2c7ce048f92242cd7278c0667f09a6d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region d2c7ce048f92242cd7278c0667f09a6d 2023-11-02 03:53:50,212 DEBUG [StoreOpener-d2c7ce048f92242cd7278c0667f09a6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1 2023-11-02 03:53:50,212 DEBUG [StoreOpener-d2c7ce048f92242cd7278c0667f09a6d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1 2023-11-02 03:53:50,213 INFO [StoreOpener-d2c7ce048f92242cd7278c0667f09a6d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d2c7ce048f92242cd7278c0667f09a6d columnFamilyName family1 2023-11-02 03:53:50,214 INFO [StoreOpener-d2c7ce048f92242cd7278c0667f09a6d-1] regionserver.HStore(310): Store=d2c7ce048f92242cd7278c0667f09a6d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:53:50,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d 2023-11-02 03:53:50,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d 2023-11-02 03:53:50,219 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d2c7ce048f92242cd7278c0667f09a6d 2023-11-02 03:53:50,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:53:50,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d2c7ce048f92242cd7278c0667f09a6d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60750081, jitterRate=-0.09475325047969818}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-02 03:53:50,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d2c7ce048f92242cd7278c0667f09a6d: 2023-11-02 03:53:50,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d., pid=25, masterSystemTime=1698897230205 2023-11-02 03:53:50,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:50,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:50,225 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=d2c7ce048f92242cd7278c0667f09a6d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:50,225 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1698897230225"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897230225"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897230225"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897230225"}]},"ts":"1698897230225"} 2023-11-02 03:53:50,228 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=25, resume processing ppid=24 2023-11-02 03:53:50,229 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure d2c7ce048f92242cd7278c0667f09a6d, server=jenkins-hbase4.apache.org,36627,1698897201574 in 174 msec 2023-11-02 03:53:50,231 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=24, resume processing ppid=23 2023-11-02 03:53:50,231 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=d2c7ce048f92242cd7278c0667f09a6d, ASSIGN in 334 msec 2023-11-02 03:53:50,236 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-02 03:53:50,237 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897230236"}]},"ts":"1698897230236"} 2023-11-02 03:53:50,238 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-11-02 03:53:50,241 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-11-02 03:53:50,242 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 1.1920 sec 2023-11-02 03:53:51,158 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-11-02 03:53:51,158 INFO [Listener at localhost/43647] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-11-02 03:53:51,184 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:51,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:51,186 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-02 03:53:51,186 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:51,186 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-02 03:53:51,186 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-02 03:53:51,186 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:51,186 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:51,192 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:51,192 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:51,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:51,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:51,192 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:51,192 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:51,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:51,193 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:51,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:51,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:51,193 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:51,193 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:51,193 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-02 03:53:51,193 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:51,194 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-02 03:53:51,194 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:51,194 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:51,194 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:51,194 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. started... 2023-11-02 03:53:51,195 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d2c7ce048f92242cd7278c0667f09a6d 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-02 03:53:51,617 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130dfeab48d2139415a81c20687d40b0db7 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151130dfeab48d2139415a81c20687d40b0db7 2023-11-02 03:53:52,023 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp/family1/f8913561839043cca5cd9c12d7d48e7a 2023-11-02 03:53:52,029 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp/family1/f8913561839043cca5cd9c12d7d48e7a as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/f8913561839043cca5cd9c12d7d48e7a 2023-11-02 03:53:52,036 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/f8913561839043cca5cd9c12d7d48e7a, entries=1000, sequenceid=5, filesize=165.1 K 2023-11-02 03:53:52,037 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for d2c7ce048f92242cd7278c0667f09a6d in 843ms, sequenceid=5, compaction requested=false 2023-11-02 03:53:52,037 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-11-02 03:53:52,038 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d2c7ce048f92242cd7278c0667f09a6d: 2023-11-02 03:53:52,038 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:52,038 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:52,038 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:52,038 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,038 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-02 03:53:52,038 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-02 03:53:52,040 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,040 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:52,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:52,040 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,041 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:52,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:52,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:52,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:52,042 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-02 03:53:52,042 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3efdb89[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:52,042 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-02 03:53:52,042 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,044 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,044 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-02 03:53:52,044 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-02 03:53:52,044 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:52,044 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,044 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:52,046 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:52,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:52,046 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-02 03:53:52,046 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:52,046 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-02 03:53:52,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:52,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:52,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:52,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:52,049 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:53:52,049 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-02 03:53:52,049 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:52,049 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:52,049 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,049 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:52,052 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,052 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,052 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:52,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:52,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:52,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:52,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:52,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:52,053 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:52,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,055 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:52,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,059 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,059 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,059 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:52,059 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,059 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,059 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,059 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,059 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,059 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-02 03:53:52,059 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:52,059 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:52,059 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:52,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:52,059 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-02 03:53:52,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:52,059 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:52,060 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:52,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:52,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:52,160 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:52,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:52,165 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:52,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:52,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,167 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-02 03:53:52,167 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:52,167 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-02 03:53:52,167 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-02 03:53:52,167 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,167 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,169 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:52,169 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:52,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:52,169 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,169 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:52,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,170 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:52,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,170 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,170 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-02 03:53:52,170 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,170 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-02 03:53:52,170 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:52,170 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-02 03:53:52,171 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:52,171 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:52,171 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:52,171 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. started... 2023-11-02 03:53:52,171 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d2c7ce048f92242cd7278c0667f09a6d 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-02 03:53:52,591 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128ab9508da71334d119c75e9959232d5be to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151128ab9508da71334d119c75e9959232d5be 2023-11-02 03:53:52,600 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp/family1/a59d528a59164ade9321b7f8f22bfa3b 2023-11-02 03:53:52,606 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp/family1/a59d528a59164ade9321b7f8f22bfa3b as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/a59d528a59164ade9321b7f8f22bfa3b 2023-11-02 03:53:52,611 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/a59d528a59164ade9321b7f8f22bfa3b, entries=1, sequenceid=9, filesize=5.0 K 2023-11-02 03:53:52,612 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for d2c7ce048f92242cd7278c0667f09a6d in 441ms, sequenceid=9, compaction requested=false 2023-11-02 03:53:52,612 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d2c7ce048f92242cd7278c0667f09a6d: 2023-11-02 03:53:52,612 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:52,612 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:52,612 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:52,612 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,612 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-02 03:53:52,612 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-02 03:53:52,618 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,618 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:52,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:52,618 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,618 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:52,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:52,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:52,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:52,620 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-02 03:53:52,620 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@30f637da[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:52,620 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-02 03:53:52,620 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,621 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,621 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-02 03:53:52,622 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-02 03:53:52,622 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:52,622 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,622 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:52,623 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,623 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-02 03:53:52,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:52,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:52,623 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:52,623 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-02 03:53:52,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:52,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:52,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:52,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:52,626 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:53:52,626 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-02 03:53:52,626 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:52,626 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:52,626 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,626 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:52,628 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,628 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:52,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:52,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,628 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:52,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:52,628 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:52,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:52,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:52,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,630 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:52,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,634 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,634 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,634 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,634 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,634 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:52,634 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,634 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-02 03:53:52,634 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:52,634 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,634 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:52,635 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:52,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:52,635 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:52,635 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:52,635 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:52,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:52,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:52,735 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:52,736 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:52,740 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:52,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:52,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,742 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-02 03:53:52,742 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:52,742 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-02 03:53:52,742 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-02 03:53:52,743 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,743 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,744 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:52,744 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:52,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:52,745 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:52,745 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:52,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,745 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:52,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,746 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,746 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-02 03:53:52,746 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:52,746 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-02 03:53:52,746 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:52,746 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-02 03:53:52,746 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:52,746 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:52,746 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:52,746 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. started... 2023-11-02 03:53:52,746 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d2c7ce048f92242cd7278c0667f09a6d 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:53,160 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120194c5ffd5de824437884a17a29949339c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120194c5ffd5de824437884a17a29949339c 2023-11-02 03:53:53,168 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp/family1/571e309145614deb84351107a7467d02 2023-11-02 03:53:53,177 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp/family1/571e309145614deb84351107a7467d02 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/571e309145614deb84351107a7467d02 2023-11-02 03:53:53,182 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/571e309145614deb84351107a7467d02, entries=1, sequenceid=13, filesize=4.9 K 2023-11-02 03:53:53,183 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d2c7ce048f92242cd7278c0667f09a6d in 437ms, sequenceid=13, compaction requested=true 2023-11-02 03:53:53,184 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d2c7ce048f92242cd7278c0667f09a6d: 2023-11-02 03:53:53,184 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:53,184 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:53,184 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:53,184 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:53,184 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-02 03:53:53,184 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-02 03:53:53,186 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:53,186 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:53,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:53,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:53,186 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,186 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:53,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:53,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:53,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:53,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:53,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-02 03:53:53,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@624c6613[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:53,188 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-02 03:53:53,188 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,189 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,189 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-02 03:53:53,189 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-02 03:53:53,190 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:53,190 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:53,190 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:53,192 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:53,192 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-02 03:53:53,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:53,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:53,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:53,192 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:53,192 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-02 03:53:53,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:53,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:53,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:53,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:53,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:53,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:53,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:53:53,194 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-02 03:53:53,194 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:53,194 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:53,194 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,194 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:53,196 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,196 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,196 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:53,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:53,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:53,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:53,197 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:53,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:53,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:53,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:53,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:53,198 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:53,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:53,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:53,200 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:53,200 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:53,200 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:53,200 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-02 03:53:53,200 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:53,200 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:53,200 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-02 03:53:53,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:53,200 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:53,200 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,201 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:53,201 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:53,201 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,201 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:53,201 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:53,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:53,201 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,301 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:53,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:53,306 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:53,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:53,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,308 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-02 03:53:53,308 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:53,308 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-02 03:53:53,308 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-02 03:53:53,308 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,308 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,310 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:53,310 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:53,310 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:53,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:53,310 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:53,310 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:53,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,310 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:53,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,311 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,311 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-02 03:53:53,311 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:53,311 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-02 03:53:53,311 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:53,311 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-02 03:53:53,311 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:53,311 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:53,311 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:53,312 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. started... 2023-11-02 03:53:53,312 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d2c7ce048f92242cd7278c0667f09a6d 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:53,728 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205139530a4bde9424bb87de25cee2613db to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205139530a4bde9424bb87de25cee2613db 2023-11-02 03:53:54,134 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp/family1/541021dd64f14c9493dde8faac0898b0 2023-11-02 03:53:54,139 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp/family1/541021dd64f14c9493dde8faac0898b0 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/541021dd64f14c9493dde8faac0898b0 2023-11-02 03:53:54,143 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/541021dd64f14c9493dde8faac0898b0, entries=1, sequenceid=17, filesize=4.9 K 2023-11-02 03:53:54,144 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d2c7ce048f92242cd7278c0667f09a6d in 832ms, sequenceid=17, compaction requested=true 2023-11-02 03:53:54,145 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d2c7ce048f92242cd7278c0667f09a6d: 2023-11-02 03:53:54,145 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:54,145 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:54,145 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:54,145 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:54,145 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-02 03:53:54,145 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-02 03:53:54,148 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:54,148 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:54,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:54,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:54,148 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,148 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:54,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:54,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:54,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:54,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:54,150 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-02 03:53:54,150 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c468d38[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:54,150 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-02 03:53:54,150 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,151 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,151 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-02 03:53:54,151 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-02 03:53:54,151 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:54,151 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:54,151 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:54,153 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:54,153 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-02 03:53:54,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:54,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:54,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:54,153 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:54,153 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-02 03:53:54,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:54,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:54,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:54,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:54,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:54,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:54,155 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:53:54,155 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-02 03:53:54,155 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:54,155 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:54,155 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,155 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:54,157 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,157 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,157 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:54,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:54,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:54,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:54,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:54,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:54,157 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:54,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:54,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,159 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:54,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:54,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:54,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:54,165 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:54,165 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,165 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,165 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:54,165 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:54,165 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-02 03:53:54,165 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:54,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:54,165 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:54,165 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-02 03:53:54,165 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:54,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:54,165 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,165 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,165 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,165 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:54,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:54,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:54,166 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:54,266 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:54,267 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:54,271 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:54,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:54,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,273 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-02 03:53:54,273 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:54,273 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-02 03:53:54,273 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-02 03:53:54,274 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,274 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,275 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:54,275 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:54,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:54,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:54,276 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:54,276 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:54,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,276 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:54,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,277 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,277 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-02 03:53:54,277 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:54,277 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-02 03:53:54,277 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:54,277 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-02 03:53:54,277 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:54,277 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:54,277 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:54,277 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. started... 2023-11-02 03:53:54,278 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d2c7ce048f92242cd7278c0667f09a6d 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:54,698 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228b9c4732be4a44407acc6639862f0c339 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228b9c4732be4a44407acc6639862f0c339 2023-11-02 03:53:55,115 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp/family1/2db93472a76046d1b0a75c7f66ff98d3 2023-11-02 03:53:55,120 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp/family1/2db93472a76046d1b0a75c7f66ff98d3 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/2db93472a76046d1b0a75c7f66ff98d3 2023-11-02 03:53:55,125 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/2db93472a76046d1b0a75c7f66ff98d3, entries=1, sequenceid=21, filesize=4.9 K 2023-11-02 03:53:55,126 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d2c7ce048f92242cd7278c0667f09a6d in 848ms, sequenceid=21, compaction requested=true 2023-11-02 03:53:55,126 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d2c7ce048f92242cd7278c0667f09a6d: 2023-11-02 03:53:55,126 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:55,126 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:55,126 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:55,126 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:55,127 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-02 03:53:55,127 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-02 03:53:55,128 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:55,128 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:55,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:55,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:55,129 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,129 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:55,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:55,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:55,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:55,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:55,130 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-02 03:53:55,130 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@110830aa[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:55,130 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-02 03:53:55,130 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,132 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,132 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-02 03:53:55,132 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-02 03:53:55,132 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:55,132 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:55,132 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:55,133 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:55,133 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-02 03:53:55,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:55,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:55,133 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:55,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:55,134 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-02 03:53:55,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:55,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:55,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:55,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:55,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:55,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:55,136 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:53:55,136 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-02 03:53:55,136 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:55,136 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:55,136 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,136 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:55,139 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,139 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:55,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:55,139 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:55,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:55,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:55,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:55,140 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:55,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:55,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:55,141 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:55,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:55,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:55,144 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:55,144 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,144 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:55,144 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:55,144 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:55,144 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-02 03:53:55,144 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:55,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:55,144 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:55,144 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:55,144 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,144 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,144 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,144 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:55,144 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:55,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:55,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:55,245 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:55,246 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:55,249 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:55,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:55,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,251 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-02 03:53:55,251 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:55,251 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-02 03:53:55,251 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-02 03:53:55,252 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,252 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,253 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:55,253 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:55,253 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:55,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:55,254 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:55,254 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:55,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,254 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:55,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,255 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,255 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-02 03:53:55,255 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:55,256 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-02 03:53:55,256 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:55,256 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-02 03:53:55,256 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:55,256 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:55,256 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:55,256 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. started... 2023-11-02 03:53:55,256 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d2c7ce048f92242cd7278c0667f09a6d 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:55,671 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231f21fd0bdd5c847268e2e862c2376d2e7 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231f21fd0bdd5c847268e2e862c2376d2e7 2023-11-02 03:53:55,750 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-02 03:53:56,077 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp/family1/af1fe33eb6634716a5fe0f8f886b295f 2023-11-02 03:53:56,083 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp/family1/af1fe33eb6634716a5fe0f8f886b295f as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/af1fe33eb6634716a5fe0f8f886b295f 2023-11-02 03:53:56,087 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/af1fe33eb6634716a5fe0f8f886b295f, entries=1, sequenceid=25, filesize=4.9 K 2023-11-02 03:53:56,088 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d2c7ce048f92242cd7278c0667f09a6d in 832ms, sequenceid=25, compaction requested=true 2023-11-02 03:53:56,088 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d2c7ce048f92242cd7278c0667f09a6d: 2023-11-02 03:53:56,088 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:56,088 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:56,089 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:56,089 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:56,089 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-02 03:53:56,089 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-02 03:53:56,090 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:56,090 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:56,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:56,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:56,091 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,091 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:56,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:56,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:56,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:56,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:56,092 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-02 03:53:56,092 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@10418db6[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:56,093 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-02 03:53:56,093 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,095 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,095 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-02 03:53:56,095 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-02 03:53:56,095 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:56,095 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:56,095 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:56,097 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:56,097 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-02 03:53:56,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:56,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:56,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:56,097 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:56,097 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-02 03:53:56,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:56,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:56,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:56,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:56,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:56,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:56,100 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:53:56,100 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-02 03:53:56,100 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:56,100 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:56,100 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,100 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:56,101 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,101 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:56,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,101 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:56,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:56,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:56,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:56,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:56,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,102 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:56,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:56,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:56,103 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:56,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:56,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:56,106 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:56,106 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,106 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:56,106 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:56,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-02 03:53:56,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:56,106 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:56,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:56,106 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-02 03:53:56,106 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:56,106 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:56,106 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,106 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,106 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,106 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:56,106 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:56,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:56,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:56,207 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:56,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:56,212 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:56,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:56,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,213 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-02 03:53:56,213 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:56,213 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-02 03:53:56,213 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-02 03:53:56,214 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,214 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,216 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:56,216 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:56,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:56,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:56,216 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:56,217 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:56,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,217 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:56,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,217 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,217 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-02 03:53:56,218 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:56,218 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-02 03:53:56,218 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:56,218 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-02 03:53:56,218 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:56,218 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:56,218 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:56,218 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. started... 2023-11-02 03:53:56,218 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d2c7ce048f92242cd7278c0667f09a6d 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:56,633 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101ba617cb84d5945cf9a6ca4077e68ca43 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101ba617cb84d5945cf9a6ca4077e68ca43 2023-11-02 03:53:57,039 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp/family1/520701b3e8f54ed394f4a75b395015d8 2023-11-02 03:53:57,044 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp/family1/520701b3e8f54ed394f4a75b395015d8 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/520701b3e8f54ed394f4a75b395015d8 2023-11-02 03:53:57,049 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/520701b3e8f54ed394f4a75b395015d8, entries=1, sequenceid=29, filesize=4.9 K 2023-11-02 03:53:57,050 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d2c7ce048f92242cd7278c0667f09a6d in 832ms, sequenceid=29, compaction requested=true 2023-11-02 03:53:57,050 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d2c7ce048f92242cd7278c0667f09a6d: 2023-11-02 03:53:57,050 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:57,050 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:57,050 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:57,050 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:57,050 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-02 03:53:57,050 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-02 03:53:57,052 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:57,052 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:57,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:57,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:57,052 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,052 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:57,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:57,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:57,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:57,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:57,054 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-02 03:53:57,054 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@11bc1d53[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:57,054 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-02 03:53:57,054 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,055 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,055 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-02 03:53:57,055 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-02 03:53:57,055 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:57,055 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:57,055 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:57,056 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:57,056 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-02 03:53:57,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:57,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:57,057 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:57,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:57,057 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-02 03:53:57,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:57,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:57,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:57,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:57,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:57,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:57,059 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:53:57,059 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-02 03:53:57,059 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:57,059 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:57,059 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,059 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:57,065 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,065 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:57,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:57,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,065 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:57,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:57,065 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:57,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:57,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:57,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:57,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:57,067 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:57,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:57,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:57,069 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:57,069 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,069 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:57,069 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:57,069 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,069 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-02 03:53:57,069 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:57,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:57,069 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:57,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:57,069 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:57,069 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,069 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:57,069 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,070 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:57,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:57,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:57,070 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,170 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:57,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:57,175 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:57,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:57,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,176 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-02 03:53:57,177 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:57,177 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-02 03:53:57,177 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-02 03:53:57,177 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,177 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,179 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:57,179 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:57,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:57,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:57,179 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:57,179 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:57,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,179 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:57,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,180 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,180 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-02 03:53:57,180 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:57,180 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-02 03:53:57,180 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:57,180 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-02 03:53:57,180 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:57,180 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:57,180 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:57,181 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. started... 2023-11-02 03:53:57,181 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d2c7ce048f92242cd7278c0667f09a6d 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:57,594 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103dfdb9904a99f4b3c8477f79d4e1a0c0b to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103dfdb9904a99f4b3c8477f79d4e1a0c0b 2023-11-02 03:53:58,000 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp/family1/bac4778ffb934caeba77a80b3b0af840 2023-11-02 03:53:58,005 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp/family1/bac4778ffb934caeba77a80b3b0af840 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/bac4778ffb934caeba77a80b3b0af840 2023-11-02 03:53:58,010 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/bac4778ffb934caeba77a80b3b0af840, entries=1, sequenceid=33, filesize=4.9 K 2023-11-02 03:53:58,011 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d2c7ce048f92242cd7278c0667f09a6d in 830ms, sequenceid=33, compaction requested=true 2023-11-02 03:53:58,011 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d2c7ce048f92242cd7278c0667f09a6d: 2023-11-02 03:53:58,011 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:58,011 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:58,011 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:58,011 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,011 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-02 03:53:58,011 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-02 03:53:58,013 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,013 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:58,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:58,013 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,013 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:58,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:58,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:58,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:58,014 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-02 03:53:58,015 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@494b7d11[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:58,015 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-02 03:53:58,015 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,017 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,017 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-02 03:53:58,017 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-02 03:53:58,017 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:58,017 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,017 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:58,018 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-02 03:53:58,018 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,018 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:58,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:58,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:58,019 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-02 03:53:58,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:58,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:58,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:58,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:58,021 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:53:58,021 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-02 03:53:58,021 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:58,021 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:58,021 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,021 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:58,023 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,023 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,023 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:58,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:58,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:58,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:58,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:58,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:58,023 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:58,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,024 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:58,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,026 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,026 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,026 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:58,026 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,027 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:58,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:58,026 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:58,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:58,027 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-02 03:53:58,027 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:58,027 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-02 03:53:58,027 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,027 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,027 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:58,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:58,027 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,027 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:58,027 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:58,127 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:58,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:58,134 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:58,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:58,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,136 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-02 03:53:58,136 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:58,136 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-02 03:53:58,136 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-02 03:53:58,137 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,137 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,139 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:58,139 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:58,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:58,140 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,140 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:58,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,140 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:58,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,141 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,141 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-02 03:53:58,141 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,141 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-02 03:53:58,141 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:58,142 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-02 03:53:58,144 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:58,144 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:58,144 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:58,144 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. started... 2023-11-02 03:53:58,144 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d2c7ce048f92242cd7278c0667f09a6d 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:58,572 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907ee3411db0e5b4bf4a600835d29edc682 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20150907ee3411db0e5b4bf4a600835d29edc682 2023-11-02 03:53:58,978 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp/family1/0a2941bd409948948e56d5e360171bbd 2023-11-02 03:53:58,983 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp/family1/0a2941bd409948948e56d5e360171bbd as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/0a2941bd409948948e56d5e360171bbd 2023-11-02 03:53:58,988 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/0a2941bd409948948e56d5e360171bbd, entries=1, sequenceid=37, filesize=4.9 K 2023-11-02 03:53:58,988 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d2c7ce048f92242cd7278c0667f09a6d in 844ms, sequenceid=37, compaction requested=true 2023-11-02 03:53:58,989 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d2c7ce048f92242cd7278c0667f09a6d: 2023-11-02 03:53:58,989 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:58,989 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:58,989 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:58,989 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,989 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-02 03:53:58,989 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-02 03:53:58,990 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,991 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:58,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:58,991 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,991 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:58,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:58,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:58,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:58,992 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-02 03:53:58,992 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@191689f2[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:58,992 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-02 03:53:58,993 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,994 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,994 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-02 03:53:58,994 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-02 03:53:58,994 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:58,994 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,994 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:58,996 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,996 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-02 03:53:58,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:58,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:58,996 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:58,996 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-02 03:53:58,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:58,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:58,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:58,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:58,999 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:53:58,999 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-02 03:53:58,999 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:58,999 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:58,999 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:58,999 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:59,002 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,002 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:59,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:59,002 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:59,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,002 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:59,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:59,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:59,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:59,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,006 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:59,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,008 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,008 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,008 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,008 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,008 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:59,009 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-02 03:53:59,009 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:59,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:59,009 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:59,009 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:59,008 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,009 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,009 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,009 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:59,009 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:59,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:59,009 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-02 03:53:59,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:59,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:59,109 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:53:59,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:53:59,114 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:53:59,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:53:59,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,116 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-11-02 03:53:59,116 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:59,116 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-02 03:53:59,116 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-02 03:53:59,117 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,117 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,118 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:59,118 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,118 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:59,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:59,119 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,119 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:53:59,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,119 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:53:59,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,120 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,120 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-11-02 03:53:59,120 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,120 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-02 03:53:59,120 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:53:59,120 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-02 03:53:59,120 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:53:59,120 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:53:59,120 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:59,120 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. started... 2023-11-02 03:53:59,120 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d2c7ce048f92242cd7278c0667f09a6d 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:53:59,536 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120e14e7ded190f4578b7e77c5976ba9d10 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151120e14e7ded190f4578b7e77c5976ba9d10 2023-11-02 03:53:59,941 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp/family1/284c51364ed9489ca58ad16704f0a3a6 2023-11-02 03:53:59,947 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp/family1/284c51364ed9489ca58ad16704f0a3a6 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/284c51364ed9489ca58ad16704f0a3a6 2023-11-02 03:53:59,951 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/284c51364ed9489ca58ad16704f0a3a6, entries=1, sequenceid=41, filesize=4.9 K 2023-11-02 03:53:59,952 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for d2c7ce048f92242cd7278c0667f09a6d in 832ms, sequenceid=41, compaction requested=true 2023-11-02 03:53:59,952 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d2c7ce048f92242cd7278c0667f09a6d: 2023-11-02 03:53:59,953 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:53:59,953 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:53:59,953 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:53:59,953 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,953 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-11-02 03:53:59,953 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-11-02 03:53:59,955 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,955 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:59,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:59,955 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,955 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:53:59,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:59,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:59,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:59,956 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-11-02 03:53:59,956 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a69b5b[Count = 0] remaining members to acquire global barrier 2023-11-02 03:53:59,956 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-02 03:53:59,956 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,962 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,962 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-02 03:53:59,962 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-11-02 03:53:59,962 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:53:59,962 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,962 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:53:59,963 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,963 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-02 03:53:59,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:59,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:59,963 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:59,963 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-11-02 03:53:59,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:59,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:59,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:59,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:53:59,967 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:53:59,967 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-11-02 03:53:59,967 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:53:59,967 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:53:59,967 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,967 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:53:59,968 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,968 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:53:59,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:53:59,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,968 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:59,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:53:59,969 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:59,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:59,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:53:59,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,970 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:53:59,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,972 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,973 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,973 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:53:59,973 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:53:59,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:53:59,973 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-11-02 03:53:59,973 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:53:59,973 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:53:59,973 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:53:59,973 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,973 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-02 03:53:59,973 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:53:59,973 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:53:59,973 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:53:59,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:53:59,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:53:59,973 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:53:59,973 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-11-02 03:54:00,073 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:00,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:00,076 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41275] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-11-02 03:54:00,078 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] procedure2.ProcedureExecutor(1028): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-02 03:54:00,079 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-02 03:54:00,081 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:54:00,081 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698897840081 2023-11-02 03:54:00,084 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-02 03:54:00,086 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-11-02 03:54:00,086 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-02 03:54:00,087 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] zookeeper.ReadOnlyZKClient(139): Connect 0x5df2dcbb to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:54:00,101 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@70e711f2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:54:00,101 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-02 03:54:00,948 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601031c9b5a92dbee44a2a4df3bf0b17a5670 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601031c9b5a92dbee44a2a4df3bf0b17a5670 2023-11-02 03:54:00,949 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-02 03:54:00,963 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-02 03:54:00,964 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56058, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-02 03:54:00,970 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-02 03:54:00,972 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49572, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-02 03:54:00,982 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/755e5a2618d54793944431c9640ee9dd first=Optional[r3] last=Optional[r6] 2023-11-02 03:54:00,982 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/755e5a2618d54793944431c9640ee9dd}] 2023-11-02 03:54:00,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/755e5a2618d54793944431c9640ee9dd for inclusion in d2c7ce048f92242cd7278c0667f09a6d/family1 2023-11-02 03:54:00,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-11-02 03:54:00,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last= 2023-11-02 03:54:00,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for d2c7ce048f92242cd7278c0667f09a6d: 2023-11-02 03:54:00,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/755e5a2618d54793944431c9640ee9dd to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__1o5m5kglvgl138597he8ce6vfl0l8k05ut3rerlaqn57a3gsivo7t8rd5uqhd6p/family1/755e5a2618d54793944431c9640ee9dd 2023-11-02 03:54:00,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__1o5m5kglvgl138597he8ce6vfl0l8k05ut3rerlaqn57a3gsivo7t8rd5uqhd6p/family1/755e5a2618d54793944431c9640ee9dd as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/6ef8a12d899c400b93c8d272f16c4ae9_SeqId_44_ 2023-11-02 03:54:00,994 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__1o5m5kglvgl138597he8ce6vfl0l8k05ut3rerlaqn57a3gsivo7t8rd5uqhd6p/family1/755e5a2618d54793944431c9640ee9dd into d2c7ce048f92242cd7278c0667f09a6d/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/6ef8a12d899c400b93c8d272f16c4ae9_SeqId_44_ - updating store file list. 2023-11-02 03:54:00,999 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/6ef8a12d899c400b93c8d272f16c4ae9_SeqId_44_ into d2c7ce048f92242cd7278c0667f09a6d/family1 2023-11-02 03:54:00,999 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__1o5m5kglvgl138597he8ce6vfl0l8k05ut3rerlaqn57a3gsivo7t8rd5uqhd6p/family1/755e5a2618d54793944431c9640ee9dd into d2c7ce048f92242cd7278c0667f09a6d/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/6ef8a12d899c400b93c8d272f16c4ae9_SeqId_44_) 2023-11-02 03:54:00,999 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__1o5m5kglvgl138597he8ce6vfl0l8k05ut3rerlaqn57a3gsivo7t8rd5uqhd6p/family1/755e5a2618d54793944431c9640ee9dd 2023-11-02 03:54:01,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-02 03:54:01,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-02 03:54:01,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(1912): d2c7ce048f92242cd7278c0667f09a6d/family1 is initiating minor compaction 2023-11-02 03:54:01,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4798505d; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:54:01,003 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d2c7ce048f92242cd7278c0667f09a6d/family1 in testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:54:01,003 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/a59d528a59164ade9321b7f8f22bfa3b, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/571e309145614deb84351107a7467d02, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/541021dd64f14c9493dde8faac0898b0, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/2db93472a76046d1b0a75c7f66ff98d3, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/af1fe33eb6634716a5fe0f8f886b295f, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/520701b3e8f54ed394f4a75b395015d8, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/bac4778ffb934caeba77a80b3b0af840, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/0a2941bd409948948e56d5e360171bbd, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/284c51364ed9489ca58ad16704f0a3a6, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/6ef8a12d899c400b93c8d272f16c4ae9_SeqId_44_] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp, totalSize=50.0 K 2023-11-02 03:54:01,003 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting a59d528a59164ade9321b7f8f22bfa3b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-02 03:54:01,004 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 571e309145614deb84351107a7467d02, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-02 03:54:01,004 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 541021dd64f14c9493dde8faac0898b0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-02 03:54:01,004 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 2db93472a76046d1b0a75c7f66ff98d3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-02 03:54:01,004 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting af1fe33eb6634716a5fe0f8f886b295f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-02 03:54:01,004 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 520701b3e8f54ed394f4a75b395015d8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-02 03:54:01,004 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting bac4778ffb934caeba77a80b3b0af840, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-02 03:54:01,004 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 0a2941bd409948948e56d5e360171bbd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-02 03:54:01,004 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 284c51364ed9489ca58ad16704f0a3a6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-02 03:54:01,004 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 6ef8a12d899c400b93c8d272f16c4ae9_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-02 03:54:01,005 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-02 03:54:01,006 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:54:01,008 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228b9c4732be4a44407acc6639862f0c339 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228b9c4732be4a44407acc6639862f0c339 2023-11-02 03:54:01,010 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231f21fd0bdd5c847268e2e862c2376d2e7 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231f21fd0bdd5c847268e2e862c2376d2e7 2023-11-02 03:54:01,011 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101ba617cb84d5945cf9a6ca4077e68ca43 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101ba617cb84d5945cf9a6ca4077e68ca43 2023-11-02 03:54:01,012 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103dfdb9904a99f4b3c8477f79d4e1a0c0b to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103dfdb9904a99f4b3c8477f79d4e1a0c0b 2023-11-02 03:54:01,012 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-11-02 03:54:01,012 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-11-02 03:54:01,013 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-02 03:54:01,013 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-02 03:54:01,013 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-02 03:54:01,013 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-02 03:54:01,013 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-02 03:54:01,013 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-02 03:54:01,013 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-02 03:54:01,013 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-02 03:54:01,049 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] throttle.PressureAwareThroughputController(145): d2c7ce048f92242cd7278c0667f09a6d#family1#compaction#34 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:54:01,083 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120575a1b07818fd4a848a5fd452edc84a06 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120575a1b07818fd4a848a5fd452edc84a06 2023-11-02 03:54:01,085 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-02 03:54:01,126 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9b9dcf45f8f94620b9f0fbdb16075339 first=Optional[r1] last=Optional[r2] 2023-11-02 03:54:01,126 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9b9dcf45f8f94620b9f0fbdb16075339}] 2023-11-02 03:54:01,131 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9b9dcf45f8f94620b9f0fbdb16075339 for inclusion in d2c7ce048f92242cd7278c0667f09a6d/family1 2023-11-02 03:54:01,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-11-02 03:54:01,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last= 2023-11-02 03:54:01,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for d2c7ce048f92242cd7278c0667f09a6d: 2023-11-02 03:54:01,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9b9dcf45f8f94620b9f0fbdb16075339 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__q4867v8cbm7mfpufkcrlbcsjspr80komka0lofpjomca8tc8l5rth7c3fm5n7g9j/family1/9b9dcf45f8f94620b9f0fbdb16075339 2023-11-02 03:54:01,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__q4867v8cbm7mfpufkcrlbcsjspr80komka0lofpjomca8tc8l5rth7c3fm5n7g9j/family1/9b9dcf45f8f94620b9f0fbdb16075339 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/24718a3254604c9da00a22c41c3c613f_SeqId_46_ 2023-11-02 03:54:01,141 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__q4867v8cbm7mfpufkcrlbcsjspr80komka0lofpjomca8tc8l5rth7c3fm5n7g9j/family1/9b9dcf45f8f94620b9f0fbdb16075339 into d2c7ce048f92242cd7278c0667f09a6d/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/24718a3254604c9da00a22c41c3c613f_SeqId_46_ - updating store file list. 2023-11-02 03:54:01,146 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/24718a3254604c9da00a22c41c3c613f_SeqId_46_ into d2c7ce048f92242cd7278c0667f09a6d/family1 2023-11-02 03:54:01,146 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__q4867v8cbm7mfpufkcrlbcsjspr80komka0lofpjomca8tc8l5rth7c3fm5n7g9j/family1/9b9dcf45f8f94620b9f0fbdb16075339 into d2c7ce048f92242cd7278c0667f09a6d/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/24718a3254604c9da00a22c41c3c613f_SeqId_46_) 2023-11-02 03:54:01,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithWeeklyPolicy__q4867v8cbm7mfpufkcrlbcsjspr80komka0lofpjomca8tc8l5rth7c3fm5n7g9j/family1/9b9dcf45f8f94620b9f0fbdb16075339 2023-11-02 03:54:01,149 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-02 03:54:01,150 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:54:01,152 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120194c5ffd5de824437884a17a29949339c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120194c5ffd5de824437884a17a29949339c 2023-11-02 03:54:01,154 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205139530a4bde9424bb87de25cee2613db to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205139530a4bde9424bb87de25cee2613db 2023-11-02 03:54:01,154 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-11-02 03:54:01,154 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-11-02 03:54:01,154 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-11-02 03:54:01,155 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-11-02 03:54:01,155 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:54:01,156 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-02 03:54:01,158 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.0790 sec 2023-11-02 03:54:01,276 INFO [Listener at localhost/43647] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=266 (was 257) - Thread LEAK? -, OpenFileDescriptor=666 (was 634) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=289 (was 344), ProcessCount=170 (was 170), AvailableMemoryMB=11341 (was 11602) 2023-11-02 03:54:01,288 INFO [Listener at localhost/43647] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=266, OpenFileDescriptor=666, MaxFileDescriptor=60000, SystemLoadAverage=289, ProcessCount=170, AvailableMemoryMB=11339 2023-11-02 03:54:01,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-11-02 03:54:01,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:01,294 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-11-02 03:54:01,294 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-11-02 03:54:01,295 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-02 03:54:01,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-11-02 03:54:01,297 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/8137557c26d83576967aea0c8ee5cd27 2023-11-02 03:54:01,298 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/bba4c1e5e128c37e7ef0161318ba5c99 2023-11-02 03:54:01,298 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b 2023-11-02 03:54:01,299 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/8137557c26d83576967aea0c8ee5cd27 empty. 2023-11-02 03:54:01,299 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/bba4c1e5e128c37e7ef0161318ba5c99 empty. 2023-11-02 03:54:01,299 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b empty. 2023-11-02 03:54:01,300 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/8137557c26d83576967aea0c8ee5cd27 2023-11-02 03:54:01,300 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b 2023-11-02 03:54:01,300 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/bba4c1e5e128c37e7ef0161318ba5c99 2023-11-02 03:54:01,300 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-11-02 03:54:01,322 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-11-02 03:54:01,324 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7675): creating {ENCODED => 8137557c26d83576967aea0c8ee5cd27, NAME => 'testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp 2023-11-02 03:54:01,326 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7675): creating {ENCODED => fadbd61541ad75cfef60a4b6222d7f6b, NAME => 'testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp 2023-11-02 03:54:01,326 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7675): creating {ENCODED => bba4c1e5e128c37e7ef0161318ba5c99, NAME => 'testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp 2023-11-02 03:54:01,350 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:01,350 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1604): Closing bba4c1e5e128c37e7ef0161318ba5c99, disabling compactions & flushes 2023-11-02 03:54:01,350 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99. 2023-11-02 03:54:01,350 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99. 2023-11-02 03:54:01,350 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99. after waiting 0 ms 2023-11-02 03:54:01,350 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99. 2023-11-02 03:54:01,350 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99. 2023-11-02 03:54:01,350 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1558): Region close journal for bba4c1e5e128c37e7ef0161318ba5c99: 2023-11-02 03:54:01,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-11-02 03:54:01,475 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/.tmp/family1/20728d6ac8464dfcafee8dd431601b0f as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/20728d6ac8464dfcafee8dd431601b0f 2023-11-02 03:54:01,482 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in d2c7ce048f92242cd7278c0667f09a6d/family1 of d2c7ce048f92242cd7278c0667f09a6d into 20728d6ac8464dfcafee8dd431601b0f(size=6.6 K), total size for store is 176.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-02 03:54:01,482 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d2c7ce048f92242cd7278c0667f09a6d: 2023-11-02 03:54:01,482 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1698897241003; duration=0sec 2023-11-02 03:54:01,482 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:54:01,569 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/a59d528a59164ade9321b7f8f22bfa3b, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/571e309145614deb84351107a7467d02, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/541021dd64f14c9493dde8faac0898b0, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/2db93472a76046d1b0a75c7f66ff98d3, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/af1fe33eb6634716a5fe0f8f886b295f, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/520701b3e8f54ed394f4a75b395015d8, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/bac4778ffb934caeba77a80b3b0af840, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/0a2941bd409948948e56d5e360171bbd, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/284c51364ed9489ca58ad16704f0a3a6, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/6ef8a12d899c400b93c8d272f16c4ae9_SeqId_44_] to archive 2023-11-02 03:54:01,570 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:54:01,572 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/a59d528a59164ade9321b7f8f22bfa3b to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/a59d528a59164ade9321b7f8f22bfa3b 2023-11-02 03:54:01,573 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/571e309145614deb84351107a7467d02 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/571e309145614deb84351107a7467d02 2023-11-02 03:54:01,575 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/541021dd64f14c9493dde8faac0898b0 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/541021dd64f14c9493dde8faac0898b0 2023-11-02 03:54:01,576 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/2db93472a76046d1b0a75c7f66ff98d3 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/2db93472a76046d1b0a75c7f66ff98d3 2023-11-02 03:54:01,577 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/af1fe33eb6634716a5fe0f8f886b295f to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/af1fe33eb6634716a5fe0f8f886b295f 2023-11-02 03:54:01,578 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/520701b3e8f54ed394f4a75b395015d8 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/520701b3e8f54ed394f4a75b395015d8 2023-11-02 03:54:01,579 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/bac4778ffb934caeba77a80b3b0af840 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/bac4778ffb934caeba77a80b3b0af840 2023-11-02 03:54:01,580 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/0a2941bd409948948e56d5e360171bbd to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/0a2941bd409948948e56d5e360171bbd 2023-11-02 03:54:01,581 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/284c51364ed9489ca58ad16704f0a3a6 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/284c51364ed9489ca58ad16704f0a3a6 2023-11-02 03:54:01,583 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/6ef8a12d899c400b93c8d272f16c4ae9_SeqId_44_ to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/family1/6ef8a12d899c400b93c8d272f16c4ae9_SeqId_44_ 2023-11-02 03:54:01,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-11-02 03:54:01,750 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:01,751 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1604): Closing 8137557c26d83576967aea0c8ee5cd27, disabling compactions & flushes 2023-11-02 03:54:01,751 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27. 2023-11-02 03:54:01,751 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27. 2023-11-02 03:54:01,751 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27. after waiting 0 ms 2023-11-02 03:54:01,751 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27. 2023-11-02 03:54:01,751 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27. 2023-11-02 03:54:01,751 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:01,751 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1558): Region close journal for 8137557c26d83576967aea0c8ee5cd27: 2023-11-02 03:54:01,751 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1604): Closing fadbd61541ad75cfef60a4b6222d7f6b, disabling compactions & flushes 2023-11-02 03:54:01,751 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b. 2023-11-02 03:54:01,751 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b. 2023-11-02 03:54:01,751 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b. after waiting 0 ms 2023-11-02 03:54:01,751 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b. 2023-11-02 03:54:01,751 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b. 2023-11-02 03:54:01,751 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1558): Region close journal for fadbd61541ad75cfef60a4b6222d7f6b: 2023-11-02 03:54:01,754 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-11-02 03:54:01,754 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1698897241754"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897241754"}]},"ts":"1698897241754"} 2023-11-02 03:54:01,755 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1698897241754"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897241754"}]},"ts":"1698897241754"} 2023-11-02 03:54:01,755 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1698897241754"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897241754"}]},"ts":"1698897241754"} 2023-11-02 03:54:01,757 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-11-02 03:54:01,758 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-02 03:54:01,758 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897241758"}]},"ts":"1698897241758"} 2023-11-02 03:54:01,760 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-11-02 03:54:01,763 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=8137557c26d83576967aea0c8ee5cd27, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=bba4c1e5e128c37e7ef0161318ba5c99, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=fadbd61541ad75cfef60a4b6222d7f6b, ASSIGN}] 2023-11-02 03:54:01,765 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=bba4c1e5e128c37e7ef0161318ba5c99, ASSIGN 2023-11-02 03:54:01,765 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=fadbd61541ad75cfef60a4b6222d7f6b, ASSIGN 2023-11-02 03:54:01,765 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=8137557c26d83576967aea0c8ee5cd27, ASSIGN 2023-11-02 03:54:01,766 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=fadbd61541ad75cfef60a4b6222d7f6b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=false 2023-11-02 03:54:01,766 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=bba4c1e5e128c37e7ef0161318ba5c99, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=false 2023-11-02 03:54:01,766 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=8137557c26d83576967aea0c8ee5cd27, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=false 2023-11-02 03:54:01,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-11-02 03:54:01,918 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=bba4c1e5e128c37e7ef0161318ba5c99, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:01,918 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=8137557c26d83576967aea0c8ee5cd27, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:01,918 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=fadbd61541ad75cfef60a4b6222d7f6b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:01,918 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1698897241918"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897241918"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897241918"}]},"ts":"1698897241918"} 2023-11-02 03:54:01,918 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1698897241918"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897241918"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897241918"}]},"ts":"1698897241918"} 2023-11-02 03:54:01,918 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1698897241918"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897241918"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897241918"}]},"ts":"1698897241918"} 2023-11-02 03:54:01,920 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=31, ppid=29, state=RUNNABLE; OpenRegionProcedure bba4c1e5e128c37e7ef0161318ba5c99, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:54:01,921 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=32, ppid=28, state=RUNNABLE; OpenRegionProcedure 8137557c26d83576967aea0c8ee5cd27, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:54:01,921 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=33, ppid=30, state=RUNNABLE; OpenRegionProcedure fadbd61541ad75cfef60a4b6222d7f6b, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:54:02,072 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:02,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27. 2023-11-02 03:54:02,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8137557c26d83576967aea0c8ee5cd27, NAME => 'testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27.', STARTKEY => '', ENDKEY => '1'} 2023-11-02 03:54:02,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 8137557c26d83576967aea0c8ee5cd27 2023-11-02 03:54:02,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:02,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8137557c26d83576967aea0c8ee5cd27 2023-11-02 03:54:02,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8137557c26d83576967aea0c8ee5cd27 2023-11-02 03:54:02,078 INFO [StoreOpener-8137557c26d83576967aea0c8ee5cd27-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8137557c26d83576967aea0c8ee5cd27 2023-11-02 03:54:02,079 DEBUG [StoreOpener-8137557c26d83576967aea0c8ee5cd27-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/8137557c26d83576967aea0c8ee5cd27/family1 2023-11-02 03:54:02,079 DEBUG [StoreOpener-8137557c26d83576967aea0c8ee5cd27-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/8137557c26d83576967aea0c8ee5cd27/family1 2023-11-02 03:54:02,080 INFO [StoreOpener-8137557c26d83576967aea0c8ee5cd27-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8137557c26d83576967aea0c8ee5cd27 columnFamilyName family1 2023-11-02 03:54:02,082 INFO [StoreOpener-8137557c26d83576967aea0c8ee5cd27-1] regionserver.HStore(310): Store=8137557c26d83576967aea0c8ee5cd27/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:54:02,082 INFO [StoreOpener-8137557c26d83576967aea0c8ee5cd27-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 8137557c26d83576967aea0c8ee5cd27 2023-11-02 03:54:02,083 DEBUG [StoreOpener-8137557c26d83576967aea0c8ee5cd27-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/8137557c26d83576967aea0c8ee5cd27/family2 2023-11-02 03:54:02,083 DEBUG [StoreOpener-8137557c26d83576967aea0c8ee5cd27-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/8137557c26d83576967aea0c8ee5cd27/family2 2023-11-02 03:54:02,084 INFO [StoreOpener-8137557c26d83576967aea0c8ee5cd27-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8137557c26d83576967aea0c8ee5cd27 columnFamilyName family2 2023-11-02 03:54:02,085 INFO [StoreOpener-8137557c26d83576967aea0c8ee5cd27-1] regionserver.HStore(310): Store=8137557c26d83576967aea0c8ee5cd27/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:54:02,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/8137557c26d83576967aea0c8ee5cd27 2023-11-02 03:54:02,087 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/8137557c26d83576967aea0c8ee5cd27 2023-11-02 03:54:02,090 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-02 03:54:02,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8137557c26d83576967aea0c8ee5cd27 2023-11-02 03:54:02,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/8137557c26d83576967aea0c8ee5cd27/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:54:02,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8137557c26d83576967aea0c8ee5cd27; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70009394, jitterRate=0.04322126507759094}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-02 03:54:02,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8137557c26d83576967aea0c8ee5cd27: 2023-11-02 03:54:02,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27., pid=32, masterSystemTime=1698897242072 2023-11-02 03:54:02,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27. 2023-11-02 03:54:02,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27. 2023-11-02 03:54:02,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b. 2023-11-02 03:54:02,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fadbd61541ad75cfef60a4b6222d7f6b, NAME => 'testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b.', STARTKEY => '2', ENDKEY => ''} 2023-11-02 03:54:02,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles fadbd61541ad75cfef60a4b6222d7f6b 2023-11-02 03:54:02,102 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=8137557c26d83576967aea0c8ee5cd27, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:02,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:02,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for fadbd61541ad75cfef60a4b6222d7f6b 2023-11-02 03:54:02,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for fadbd61541ad75cfef60a4b6222d7f6b 2023-11-02 03:54:02,103 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1698897242102"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897242102"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897242102"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897242102"}]},"ts":"1698897242102"} 2023-11-02 03:54:02,104 INFO [StoreOpener-fadbd61541ad75cfef60a4b6222d7f6b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region fadbd61541ad75cfef60a4b6222d7f6b 2023-11-02 03:54:02,105 DEBUG [StoreOpener-fadbd61541ad75cfef60a4b6222d7f6b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1 2023-11-02 03:54:02,105 DEBUG [StoreOpener-fadbd61541ad75cfef60a4b6222d7f6b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1 2023-11-02 03:54:02,106 INFO [StoreOpener-fadbd61541ad75cfef60a4b6222d7f6b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fadbd61541ad75cfef60a4b6222d7f6b columnFamilyName family1 2023-11-02 03:54:02,106 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=32, resume processing ppid=28 2023-11-02 03:54:02,106 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=32, ppid=28, state=SUCCESS; OpenRegionProcedure 8137557c26d83576967aea0c8ee5cd27, server=jenkins-hbase4.apache.org,36627,1698897201574 in 183 msec 2023-11-02 03:54:02,107 INFO [StoreOpener-fadbd61541ad75cfef60a4b6222d7f6b-1] regionserver.HStore(310): Store=fadbd61541ad75cfef60a4b6222d7f6b/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:54:02,107 INFO [StoreOpener-fadbd61541ad75cfef60a4b6222d7f6b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region fadbd61541ad75cfef60a4b6222d7f6b 2023-11-02 03:54:02,108 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=8137557c26d83576967aea0c8ee5cd27, ASSIGN in 344 msec 2023-11-02 03:54:02,109 DEBUG [StoreOpener-fadbd61541ad75cfef60a4b6222d7f6b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family2 2023-11-02 03:54:02,109 DEBUG [StoreOpener-fadbd61541ad75cfef60a4b6222d7f6b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family2 2023-11-02 03:54:02,109 INFO [StoreOpener-fadbd61541ad75cfef60a4b6222d7f6b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fadbd61541ad75cfef60a4b6222d7f6b columnFamilyName family2 2023-11-02 03:54:02,110 INFO [StoreOpener-fadbd61541ad75cfef60a4b6222d7f6b-1] regionserver.HStore(310): Store=fadbd61541ad75cfef60a4b6222d7f6b/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:54:02,111 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b 2023-11-02 03:54:02,111 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b 2023-11-02 03:54:02,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-02 03:54:02,114 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for fadbd61541ad75cfef60a4b6222d7f6b 2023-11-02 03:54:02,116 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:54:02,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened fadbd61541ad75cfef60a4b6222d7f6b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72147727, jitterRate=0.07508490979671478}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-02 03:54:02,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for fadbd61541ad75cfef60a4b6222d7f6b: 2023-11-02 03:54:02,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b., pid=33, masterSystemTime=1698897242072 2023-11-02 03:54:02,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b. 2023-11-02 03:54:02,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b. 2023-11-02 03:54:02,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99. 2023-11-02 03:54:02,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bba4c1e5e128c37e7ef0161318ba5c99, NAME => 'testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99.', STARTKEY => '1', ENDKEY => '2'} 2023-11-02 03:54:02,120 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles bba4c1e5e128c37e7ef0161318ba5c99 2023-11-02 03:54:02,120 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=fadbd61541ad75cfef60a4b6222d7f6b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:02,120 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:02,120 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for bba4c1e5e128c37e7ef0161318ba5c99 2023-11-02 03:54:02,120 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1698897242119"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897242119"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897242119"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897242119"}]},"ts":"1698897242119"} 2023-11-02 03:54:02,120 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for bba4c1e5e128c37e7ef0161318ba5c99 2023-11-02 03:54:02,121 INFO [StoreOpener-bba4c1e5e128c37e7ef0161318ba5c99-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region bba4c1e5e128c37e7ef0161318ba5c99 2023-11-02 03:54:02,123 DEBUG [StoreOpener-bba4c1e5e128c37e7ef0161318ba5c99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/bba4c1e5e128c37e7ef0161318ba5c99/family1 2023-11-02 03:54:02,123 DEBUG [StoreOpener-bba4c1e5e128c37e7ef0161318ba5c99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/bba4c1e5e128c37e7ef0161318ba5c99/family1 2023-11-02 03:54:02,123 INFO [StoreOpener-bba4c1e5e128c37e7ef0161318ba5c99-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bba4c1e5e128c37e7ef0161318ba5c99 columnFamilyName family1 2023-11-02 03:54:02,123 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=33, resume processing ppid=30 2023-11-02 03:54:02,123 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=33, ppid=30, state=SUCCESS; OpenRegionProcedure fadbd61541ad75cfef60a4b6222d7f6b, server=jenkins-hbase4.apache.org,36627,1698897201574 in 200 msec 2023-11-02 03:54:02,124 INFO [StoreOpener-bba4c1e5e128c37e7ef0161318ba5c99-1] regionserver.HStore(310): Store=bba4c1e5e128c37e7ef0161318ba5c99/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:54:02,125 INFO [StoreOpener-bba4c1e5e128c37e7ef0161318ba5c99-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region bba4c1e5e128c37e7ef0161318ba5c99 2023-11-02 03:54:02,125 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=fadbd61541ad75cfef60a4b6222d7f6b, ASSIGN in 360 msec 2023-11-02 03:54:02,126 DEBUG [StoreOpener-bba4c1e5e128c37e7ef0161318ba5c99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/bba4c1e5e128c37e7ef0161318ba5c99/family2 2023-11-02 03:54:02,126 DEBUG [StoreOpener-bba4c1e5e128c37e7ef0161318ba5c99-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/bba4c1e5e128c37e7ef0161318ba5c99/family2 2023-11-02 03:54:02,127 INFO [StoreOpener-bba4c1e5e128c37e7ef0161318ba5c99-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bba4c1e5e128c37e7ef0161318ba5c99 columnFamilyName family2 2023-11-02 03:54:02,127 INFO [StoreOpener-bba4c1e5e128c37e7ef0161318ba5c99-1] regionserver.HStore(310): Store=bba4c1e5e128c37e7ef0161318ba5c99/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:54:02,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/bba4c1e5e128c37e7ef0161318ba5c99 2023-11-02 03:54:02,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/bba4c1e5e128c37e7ef0161318ba5c99 2023-11-02 03:54:02,131 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-02 03:54:02,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for bba4c1e5e128c37e7ef0161318ba5c99 2023-11-02 03:54:02,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/bba4c1e5e128c37e7ef0161318ba5c99/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:54:02,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened bba4c1e5e128c37e7ef0161318ba5c99; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62769131, jitterRate=-0.06466706097126007}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-02 03:54:02,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for bba4c1e5e128c37e7ef0161318ba5c99: 2023-11-02 03:54:02,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99., pid=31, masterSystemTime=1698897242072 2023-11-02 03:54:02,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99. 2023-11-02 03:54:02,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99. 2023-11-02 03:54:02,136 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=bba4c1e5e128c37e7ef0161318ba5c99, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:02,137 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1698897242136"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897242136"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897242136"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897242136"}]},"ts":"1698897242136"} 2023-11-02 03:54:02,140 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=31, resume processing ppid=29 2023-11-02 03:54:02,140 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=31, ppid=29, state=SUCCESS; OpenRegionProcedure bba4c1e5e128c37e7ef0161318ba5c99, server=jenkins-hbase4.apache.org,36627,1698897201574 in 218 msec 2023-11-02 03:54:02,141 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=29, resume processing ppid=27 2023-11-02 03:54:02,142 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=bba4c1e5e128c37e7ef0161318ba5c99, ASSIGN in 377 msec 2023-11-02 03:54:02,142 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-02 03:54:02,142 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897242142"}]},"ts":"1698897242142"} 2023-11-02 03:54:02,143 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-11-02 03:54:02,146 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-11-02 03:54:02,147 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 855 msec 2023-11-02 03:54:02,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=27 2023-11-02 03:54:02,400 INFO [Listener at localhost/43647] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-11-02 03:54:02,408 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:02,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,410 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-11-02 03:54:02,410 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:02,410 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-11-02 03:54:02,410 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-11-02 03:54:02,411 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,411 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,412 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:02,412 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:02,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:02,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:02,413 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:02,413 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:02,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,413 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:02,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,414 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,414 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,414 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-11-02 03:54:02,414 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:02,414 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-11-02 03:54:02,414 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99. 2023-11-02 03:54:02,414 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b. 2023-11-02 03:54:02,414 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-02 03:54:02,415 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:02,414 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99. started... 2023-11-02 03:54:02,415 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27. 2023-11-02 03:54:02,415 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b. started... 2023-11-02 03:54:02,415 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27. started... 2023-11-02 03:54:02,415 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bba4c1e5e128c37e7ef0161318ba5c99: 2023-11-02 03:54:02,416 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing fadbd61541ad75cfef60a4b6222d7f6b 2/2 column families, dataSize=48 B heapSize=632 B 2023-11-02 03:54:02,416 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 8137557c26d83576967aea0c8ee5cd27: 2023-11-02 03:54:02,416 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99. 2023-11-02 03:54:02,417 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27. 2023-11-02 03:54:02,417 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-02 03:54:02,417 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-02 03:54:02,838 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231102921eddd0eb3c463d95b247c57a1c93eb to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20231102921eddd0eb3c463d95b247c57a1c93eb 2023-11-02 03:54:02,852 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/.tmp/family1/dd4a2f38e1fc428990c7b2c38a9867fc 2023-11-02 03:54:02,888 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-02 03:54:02,888 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x685d2806 to 127.0.0.1:65185 2023-11-02 03:54:02,888 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:54:02,890 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-02 03:54:02,890 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x37eaf16e to 127.0.0.1:65185 2023-11-02 03:54:02,890 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:54:02,891 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-02 03:54:02,891 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6b262b5d to 127.0.0.1:65185 2023-11-02 03:54:02,891 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:54:02,891 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-02 03:54:02,900 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5df2dcbb to 127.0.0.1:65185 2023-11-02 03:54:02,900 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:54:02,902 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/.tmp/family1/dd4a2f38e1fc428990c7b2c38a9867fc as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1/dd4a2f38e1fc428990c7b2c38a9867fc 2023-11-02 03:54:02,912 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1/dd4a2f38e1fc428990c7b2c38a9867fc, entries=1, sequenceid=5, filesize=4.9 K 2023-11-02 03:54:02,914 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for fadbd61541ad75cfef60a4b6222d7f6b in 498ms, sequenceid=5, compaction requested=false 2023-11-02 03:54:02,914 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-11-02 03:54:02,915 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for fadbd61541ad75cfef60a4b6222d7f6b: 2023-11-02 03:54:02,915 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b. 2023-11-02 03:54:02,915 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-02 03:54:02,915 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-02 03:54:02,915 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:02,915 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-11-02 03:54:02,915 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-11-02 03:54:02,917 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:02,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:02,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:02,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:02,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:02,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:02,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:02,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:02,919 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-11-02 03:54:02,919 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74015839[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:02,919 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-11-02 03:54:02,920 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,921 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,921 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(162): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,921 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,921 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:02,921 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-11-02 03:54:02,922 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-11-02 03:54:02,922 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:02,922 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:02,922 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:02,925 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-11-02 03:54:02,925 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:02,925 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-11-02 03:54:02,925 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:02,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:02,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:02,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:02,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:02,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:02,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:02,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:02,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:02,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:02,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-11-02 03:54:02,928 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-11-02 03:54:02,929 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:02,929 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:02,929 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,929 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:02,931 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,931 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:02,931 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:02,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:02,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:02,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:02,931 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:02,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,933 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:02,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:02,935 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:02,935 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,935 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:02,935 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-11-02 03:54:02,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:02,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:02,936 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:02,936 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,936 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,936 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:02,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:02,936 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:02,936 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:02,936 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:02,936 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:02,936 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:02,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:02,938 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-11-02 03:54:02,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:02,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:03,037 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:03,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:03,042 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:03,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:03,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,044 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-11-02 03:54:03,044 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:03,044 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-11-02 03:54:03,044 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-11-02 03:54:03,044 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,044 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,045 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:03,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:03,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:03,046 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:03,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,046 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:03,046 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:03,046 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:03,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,047 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,047 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-11-02 03:54:03,047 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,047 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-11-02 03:54:03,047 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:03,047 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-11-02 03:54:03,048 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99. 2023-11-02 03:54:03,048 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b. 2023-11-02 03:54:03,048 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-02 03:54:03,048 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:03,048 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99. started... 2023-11-02 03:54:03,048 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27. 2023-11-02 03:54:03,048 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b. started... 2023-11-02 03:54:03,048 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for bba4c1e5e128c37e7ef0161318ba5c99: 2023-11-02 03:54:03,048 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99. 2023-11-02 03:54:03,048 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27. started... 2023-11-02 03:54:03,048 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-02 03:54:03,048 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing fadbd61541ad75cfef60a4b6222d7f6b 2/2 column families, dataSize=90 B heapSize=752 B 2023-11-02 03:54:03,048 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 8137557c26d83576967aea0c8ee5cd27: 2023-11-02 03:54:03,048 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27. 2023-11-02 03:54:03,048 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-02 03:54:03,466 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231102f83240b6ef654ea8a3145519a1e19de9 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20231102f83240b6ef654ea8a3145519a1e19de9 2023-11-02 03:54:03,472 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/.tmp/family1/d8c480b9c97d4d19843bf383d93b1cdc 2023-11-02 03:54:03,477 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/.tmp/family1/d8c480b9c97d4d19843bf383d93b1cdc as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1/d8c480b9c97d4d19843bf383d93b1cdc 2023-11-02 03:54:03,482 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1/d8c480b9c97d4d19843bf383d93b1cdc, entries=2, sequenceid=9, filesize=5.0 K 2023-11-02 03:54:03,483 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for fadbd61541ad75cfef60a4b6222d7f6b in 435ms, sequenceid=9, compaction requested=false 2023-11-02 03:54:03,483 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for fadbd61541ad75cfef60a4b6222d7f6b: 2023-11-02 03:54:03,484 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b. 2023-11-02 03:54:03,484 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-02 03:54:03,484 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-02 03:54:03,484 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:03,484 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-11-02 03:54:03,484 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-11-02 03:54:03,487 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:03,487 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:03,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:03,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:03,487 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,487 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:03,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:03,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:03,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:03,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:03,489 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-11-02 03:54:03,489 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6630005d[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:03,489 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-11-02 03:54:03,489 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,490 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,491 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-11-02 03:54:03,491 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-11-02 03:54:03,491 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:03,491 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:03,491 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:03,492 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:03,492 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-11-02 03:54:03,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:03,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:03,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:03,492 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:03,493 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-11-02 03:54:03,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:03,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:03,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:03,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:03,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:03,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:03,495 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-11-02 03:54:03,495 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-11-02 03:54:03,495 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:03,495 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:03,495 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,495 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:03,504 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,504 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:03,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:03,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,505 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:03,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:03,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:03,505 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:03,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:03,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:03,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:03,507 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:03,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:03,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:03,510 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:03,511 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:03,511 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:03,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:03,511 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:03,511 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,511 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:03,511 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:03,511 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-11-02 03:54:03,511 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:03,511 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,511 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,511 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:03,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:03,511 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:03,512 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:03,612 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:03,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:03,619 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41275] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-11-02 03:54:03,621 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] procedure2.ProcedureExecutor(1028): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-11-02 03:54:03,621 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-11-02 03:54:03,622 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:54:03,622 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698897843622 2023-11-02 03:54:03,623 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-02 03:54:03,623 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-11-02 03:54:03,623 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-11-02 03:54:03,624 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] zookeeper.ReadOnlyZKClient(139): Connect 0x5a405a6f to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:54:03,628 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@12e0a854, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:54:03,628 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231102 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-11-02 03:54:04,454 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231102a2280c93edd64ac180e279123b0af949 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20231102a2280c93edd64ac180e279123b0af949 2023-11-02 03:54:04,455 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231102 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-11-02 03:54:04,471 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-02 03:54:04,472 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56068, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-02 03:54:04,479 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-02 03:54:04,481 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49584, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-02 03:54:04,490 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231102/family1/5f701c0aae074e5b96ead2b8fbb00581 first=Optional[k0] last=Optional[k1] 2023-11-02 03:54:04,490 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231102/family1/5f701c0aae074e5b96ead2b8fbb00581}] 2023-11-02 03:54:04,495 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231102/family1/5f701c0aae074e5b96ead2b8fbb00581 for inclusion in fadbd61541ad75cfef60a4b6222d7f6b/family1 2023-11-02 03:54:04,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-11-02 03:54:04,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first=2 last= 2023-11-02 03:54:04,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for fadbd61541ad75cfef60a4b6222d7f6b: 2023-11-02 03:54:04,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231102/family1/5f701c0aae074e5b96ead2b8fbb00581 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testScannerOnBulkLoadRefHFiles__nr344ddqbmv95cpmslasndfvumherj3gedic62jo9s0at14jgpoed6ae39qe8g7e/family1/5f701c0aae074e5b96ead2b8fbb00581 2023-11-02 03:54:04,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testScannerOnBulkLoadRefHFiles__nr344ddqbmv95cpmslasndfvumherj3gedic62jo9s0at14jgpoed6ae39qe8g7e/family1/5f701c0aae074e5b96ead2b8fbb00581 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1/c523d6e4197e4752b338fcb0ba0fadf9_SeqId_12_ 2023-11-02 03:54:04,501 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testScannerOnBulkLoadRefHFiles__nr344ddqbmv95cpmslasndfvumherj3gedic62jo9s0at14jgpoed6ae39qe8g7e/family1/5f701c0aae074e5b96ead2b8fbb00581 into fadbd61541ad75cfef60a4b6222d7f6b/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1/c523d6e4197e4752b338fcb0ba0fadf9_SeqId_12_ - updating store file list. 2023-11-02 03:54:04,505 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1/c523d6e4197e4752b338fcb0ba0fadf9_SeqId_12_ into fadbd61541ad75cfef60a4b6222d7f6b/family1 2023-11-02 03:54:04,505 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testScannerOnBulkLoadRefHFiles__nr344ddqbmv95cpmslasndfvumherj3gedic62jo9s0at14jgpoed6ae39qe8g7e/family1/5f701c0aae074e5b96ead2b8fbb00581 into fadbd61541ad75cfef60a4b6222d7f6b/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1/c523d6e4197e4752b338fcb0ba0fadf9_SeqId_12_) 2023-11-02 03:54:04,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testScannerOnBulkLoadRefHFiles__nr344ddqbmv95cpmslasndfvumherj3gedic62jo9s0at14jgpoed6ae39qe8g7e/family1/5f701c0aae074e5b96ead2b8fbb00581 2023-11-02 03:54:04,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-02 03:54:04,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-02 03:54:04,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(1912): fadbd61541ad75cfef60a4b6222d7f6b/family1 is initiating minor compaction (all files) 2023-11-02 03:54:04,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@8b17e71; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:54:04,507 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of fadbd61541ad75cfef60a4b6222d7f6b/family1 in testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b. 2023-11-02 03:54:04,508 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1/dd4a2f38e1fc428990c7b2c38a9867fc, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1/d8c480b9c97d4d19843bf383d93b1cdc, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1/c523d6e4197e4752b338fcb0ba0fadf9_SeqId_12_] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/.tmp, totalSize=15.1 K 2023-11-02 03:54:04,509 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting dd4a2f38e1fc428990c7b2c38a9867fc, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897242400 2023-11-02 03:54:04,511 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting d8c480b9c97d4d19843bf383d93b1cdc, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897242400 2023-11-02 03:54:04,511 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting c523d6e4197e4752b338fcb0ba0fadf9_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897242400 2023-11-02 03:54:04,511 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231102 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-11-02 03:54:04,512 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:54:04,514 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20231102921eddd0eb3c463d95b247c57a1c93eb to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20231102921eddd0eb3c463d95b247c57a1c93eb 2023-11-02 03:54:04,516 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20231102f83240b6ef654ea8a3145519a1e19de9 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20231102f83240b6ef654ea8a3145519a1e19de9 2023-11-02 03:54:04,516 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-11-02 03:54:04,516 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20231102' 2023-11-02 03:54:04,516 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231102 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-11-02 03:54:04,516 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-11-02 03:54:04,516 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:54:04,518 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-11-02 03:54:04,520 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 898 msec 2023-11-02 03:54:04,533 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] throttle.PressureAwareThroughputController(145): fadbd61541ad75cfef60a4b6222d7f6b#family1#compaction#37 average throughput is 0.05 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:54:04,560 INFO [Listener at localhost/43647] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=263 (was 266), OpenFileDescriptor=644 (was 666), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=266 (was 289), ProcessCount=170 (was 170), AvailableMemoryMB=11270 (was 11339) 2023-11-02 03:54:04,576 INFO [Listener at localhost/43647] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=260, OpenFileDescriptor=642, MaxFileDescriptor=60000, SystemLoadAverage=266, ProcessCount=170, AvailableMemoryMB=11270 2023-11-02 03:54:04,578 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-02 03:54:04,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:04,581 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-11-02 03:54:04,581 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-11-02 03:54:04,582 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-02 03:54:04,582 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-02 03:54:04,584 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73 2023-11-02 03:54:04,584 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73 empty. 2023-11-02 03:54:04,585 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73 2023-11-02 03:54:04,585 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-11-02 03:54:04,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-02 03:54:04,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-02 03:54:04,995 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-11-02 03:54:04,997 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => cb0f3da5a940148bb91c8b121c586d73, NAME => 'testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp 2023-11-02 03:54:05,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-02 03:54:05,366 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/.tmp/family1/1dceddec0d544489bcf615c1c3a4886a as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1/1dceddec0d544489bcf615c1c3a4886a 2023-11-02 03:54:05,372 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in fadbd61541ad75cfef60a4b6222d7f6b/family1 of fadbd61541ad75cfef60a4b6222d7f6b into 1dceddec0d544489bcf615c1c3a4886a(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-02 03:54:05,372 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for fadbd61541ad75cfef60a4b6222d7f6b: 2023-11-02 03:54:05,372 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1698897244507; duration=0sec 2023-11-02 03:54:05,372 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:54:05,403 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:05,403 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1604): Closing cb0f3da5a940148bb91c8b121c586d73, disabling compactions & flushes 2023-11-02 03:54:05,403 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:05,403 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:05,403 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. after waiting 0 ms 2023-11-02 03:54:05,403 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:05,403 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:05,403 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for cb0f3da5a940148bb91c8b121c586d73: 2023-11-02 03:54:05,405 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-11-02 03:54:05,406 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1698897245406"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897245406"}]},"ts":"1698897245406"} 2023-11-02 03:54:05,408 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-02 03:54:05,408 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-02 03:54:05,409 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897245408"}]},"ts":"1698897245408"} 2023-11-02 03:54:05,410 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-11-02 03:54:05,413 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=cb0f3da5a940148bb91c8b121c586d73, ASSIGN}] 2023-11-02 03:54:05,414 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=cb0f3da5a940148bb91c8b121c586d73, ASSIGN 2023-11-02 03:54:05,414 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=cb0f3da5a940148bb91c8b121c586d73, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=false 2023-11-02 03:54:05,453 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1/dd4a2f38e1fc428990c7b2c38a9867fc, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1/d8c480b9c97d4d19843bf383d93b1cdc, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1/c523d6e4197e4752b338fcb0ba0fadf9_SeqId_12_] to archive 2023-11-02 03:54:05,454 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:54:05,456 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1/dd4a2f38e1fc428990c7b2c38a9867fc to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1/dd4a2f38e1fc428990c7b2c38a9867fc 2023-11-02 03:54:05,457 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1/d8c480b9c97d4d19843bf383d93b1cdc to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1/d8c480b9c97d4d19843bf383d93b1cdc 2023-11-02 03:54:05,458 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1/c523d6e4197e4752b338fcb0ba0fadf9_SeqId_12_ to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/family1/c523d6e4197e4752b338fcb0ba0fadf9_SeqId_12_ 2023-11-02 03:54:05,565 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=cb0f3da5a940148bb91c8b121c586d73, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:05,566 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1698897245565"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897245565"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897245565"}]},"ts":"1698897245565"} 2023-11-02 03:54:05,567 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure cb0f3da5a940148bb91c8b121c586d73, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:54:05,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-02 03:54:05,719 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:05,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:05,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cb0f3da5a940148bb91c8b121c586d73, NAME => 'testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73.', STARTKEY => '', ENDKEY => ''} 2023-11-02 03:54:05,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy cb0f3da5a940148bb91c8b121c586d73 2023-11-02 03:54:05,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:05,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for cb0f3da5a940148bb91c8b121c586d73 2023-11-02 03:54:05,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for cb0f3da5a940148bb91c8b121c586d73 2023-11-02 03:54:05,723 INFO [StoreOpener-cb0f3da5a940148bb91c8b121c586d73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region cb0f3da5a940148bb91c8b121c586d73 2023-11-02 03:54:05,725 DEBUG [StoreOpener-cb0f3da5a940148bb91c8b121c586d73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1 2023-11-02 03:54:05,725 DEBUG [StoreOpener-cb0f3da5a940148bb91c8b121c586d73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1 2023-11-02 03:54:05,725 INFO [StoreOpener-cb0f3da5a940148bb91c8b121c586d73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cb0f3da5a940148bb91c8b121c586d73 columnFamilyName family1 2023-11-02 03:54:05,726 INFO [StoreOpener-cb0f3da5a940148bb91c8b121c586d73-1] regionserver.HStore(310): Store=cb0f3da5a940148bb91c8b121c586d73/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:54:05,727 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73 2023-11-02 03:54:05,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73 2023-11-02 03:54:05,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for cb0f3da5a940148bb91c8b121c586d73 2023-11-02 03:54:05,732 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:54:05,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened cb0f3da5a940148bb91c8b121c586d73; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67951871, jitterRate=0.012561783194541931}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-02 03:54:05,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for cb0f3da5a940148bb91c8b121c586d73: 2023-11-02 03:54:05,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73., pid=37, masterSystemTime=1698897245719 2023-11-02 03:54:05,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:05,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:05,736 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=cb0f3da5a940148bb91c8b121c586d73, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:05,736 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1698897245736"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897245736"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897245736"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897245736"}]},"ts":"1698897245736"} 2023-11-02 03:54:05,739 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=37, resume processing ppid=36 2023-11-02 03:54:05,739 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure cb0f3da5a940148bb91c8b121c586d73, server=jenkins-hbase4.apache.org,36627,1698897201574 in 170 msec 2023-11-02 03:54:05,741 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=36, resume processing ppid=35 2023-11-02 03:54:05,741 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=cb0f3da5a940148bb91c8b121c586d73, ASSIGN in 327 msec 2023-11-02 03:54:05,741 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-02 03:54:05,741 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897245741"}]},"ts":"1698897245741"} 2023-11-02 03:54:05,743 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-11-02 03:54:05,745 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-11-02 03:54:05,747 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 1.1670 sec 2023-11-02 03:54:06,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-11-02 03:54:06,688 INFO [Listener at localhost/43647] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-11-02 03:54:06,714 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:06,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:06,716 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-02 03:54:06,716 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:06,716 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-02 03:54:06,716 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-02 03:54:06,716 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:06,716 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:06,719 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:06,719 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:06,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:06,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:06,720 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:06,720 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:06,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:06,720 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:06,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:06,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:06,721 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:06,721 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:06,721 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-02 03:54:06,721 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:06,721 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-02 03:54:06,721 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:06,721 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:06,721 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:06,721 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. started... 2023-11-02 03:54:06,722 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cb0f3da5a940148bb91c8b121c586d73 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-02 03:54:07,140 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130b45d62b653d641089cde138a64234e08 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130b45d62b653d641089cde138a64234e08 2023-11-02 03:54:07,544 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/618237a641134f49be947204037abfca 2023-11-02 03:54:07,549 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/618237a641134f49be947204037abfca as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/618237a641134f49be947204037abfca 2023-11-02 03:54:07,553 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/618237a641134f49be947204037abfca, entries=1000, sequenceid=5, filesize=165.1 K 2023-11-02 03:54:07,554 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for cb0f3da5a940148bb91c8b121c586d73 in 832ms, sequenceid=5, compaction requested=false 2023-11-02 03:54:07,554 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-11-02 03:54:07,555 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cb0f3da5a940148bb91c8b121c586d73: 2023-11-02 03:54:07,555 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:07,555 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:07,555 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:07,555 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:07,555 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-02 03:54:07,555 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-02 03:54:07,557 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:07,557 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:07,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:07,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:07,557 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,557 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:07,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:07,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:07,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:07,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:07,559 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-02 03:54:07,559 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4bc45c70[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:07,559 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-02 03:54:07,559 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,560 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,560 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-02 03:54:07,560 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-02 03:54:07,560 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:07,560 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:07,560 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:07,561 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:07,561 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-02 03:54:07,561 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:07,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:07,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:07,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:07,561 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-02 03:54:07,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:07,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:07,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:07,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:07,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:07,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:07,564 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:54:07,564 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-02 03:54:07,564 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:07,564 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:07,564 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,564 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:07,567 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,567 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:07,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,567 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:07,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:07,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:07,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:07,568 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:07,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:07,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:07,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,569 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:07,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:07,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:07,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:07,571 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:07,571 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,571 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:07,571 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:07,571 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:07,571 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-02 03:54:07,571 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:07,571 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-02 03:54:07,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:07,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:07,571 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:07,571 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,571 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,572 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:07,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:07,572 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:07,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:07,603 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-02 03:54:07,672 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:07,673 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:07,677 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:07,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:07,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,678 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-02 03:54:07,678 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:07,678 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-02 03:54:07,678 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-02 03:54:07,679 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,679 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,680 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:07,680 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:07,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:07,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:07,680 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:07,680 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:07,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,681 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:07,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,681 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,681 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-02 03:54:07,681 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:07,681 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-02 03:54:07,682 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:07,682 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-02 03:54:07,682 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:07,682 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:07,682 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:07,682 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. started... 2023-11-02 03:54:07,682 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cb0f3da5a940148bb91c8b121c586d73 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-02 03:54:08,095 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128e5b4e6a999c041a7a2f20e088074feae to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151128e5b4e6a999c041a7a2f20e088074feae 2023-11-02 03:54:08,500 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/211b43445f4a4b08be6383c73d0f2794 2023-11-02 03:54:08,505 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/211b43445f4a4b08be6383c73d0f2794 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/211b43445f4a4b08be6383c73d0f2794 2023-11-02 03:54:08,509 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/211b43445f4a4b08be6383c73d0f2794, entries=1, sequenceid=9, filesize=5.0 K 2023-11-02 03:54:08,510 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for cb0f3da5a940148bb91c8b121c586d73 in 828ms, sequenceid=9, compaction requested=false 2023-11-02 03:54:08,510 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cb0f3da5a940148bb91c8b121c586d73: 2023-11-02 03:54:08,510 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:08,510 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:08,510 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:08,510 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:08,510 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-02 03:54:08,510 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-02 03:54:08,512 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:08,512 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:08,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:08,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:08,512 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,512 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:08,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:08,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:08,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:08,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:08,514 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-02 03:54:08,514 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a096a10[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:08,514 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-02 03:54:08,514 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,515 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,516 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-02 03:54:08,516 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-02 03:54:08,516 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:08,516 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:08,516 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:08,517 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-02 03:54:08,517 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:08,517 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:08,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:08,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:08,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:08,517 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-02 03:54:08,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:08,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:08,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:08,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:08,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:08,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:08,520 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:54:08,520 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-02 03:54:08,521 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:08,521 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:08,521 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,521 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:08,523 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,523 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:08,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,523 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:08,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:08,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:08,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:08,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:08,523 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:08,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:08,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,525 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:08,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:08,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:08,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:08,532 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:08,532 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,532 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:08,532 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:08,532 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-02 03:54:08,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:08,532 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-02 03:54:08,532 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:08,532 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:08,532 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:08,532 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:08,532 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,533 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,533 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:08,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:08,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:08,533 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,533 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:08,633 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:08,634 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:08,638 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:08,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:08,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,640 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-02 03:54:08,640 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:08,640 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-02 03:54:08,640 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-02 03:54:08,640 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,640 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,642 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:08,642 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:08,642 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:08,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:08,642 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:08,642 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:08,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,642 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:08,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,643 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,643 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-02 03:54:08,643 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:08,643 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-02 03:54:08,643 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:08,643 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-02 03:54:08,643 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:08,643 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:08,643 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:08,643 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. started... 2023-11-02 03:54:08,643 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cb0f3da5a940148bb91c8b121c586d73 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:09,056 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120149882fcc3f8a49419c065a0c9bff2f0a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120149882fcc3f8a49419c065a0c9bff2f0a 2023-11-02 03:54:09,062 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/97931e699be44669b8acdeadf644b7c0 2023-11-02 03:54:09,066 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/97931e699be44669b8acdeadf644b7c0 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/97931e699be44669b8acdeadf644b7c0 2023-11-02 03:54:09,070 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/97931e699be44669b8acdeadf644b7c0, entries=1, sequenceid=13, filesize=4.9 K 2023-11-02 03:54:09,071 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb0f3da5a940148bb91c8b121c586d73 in 428ms, sequenceid=13, compaction requested=true 2023-11-02 03:54:09,071 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cb0f3da5a940148bb91c8b121c586d73: 2023-11-02 03:54:09,071 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:09,071 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:09,071 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:09,071 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:09,071 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-02 03:54:09,071 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-02 03:54:09,073 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:09,073 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:09,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:09,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:09,073 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,073 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:09,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:09,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:09,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:09,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:09,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-02 03:54:09,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b725f8[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:09,075 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-02 03:54:09,075 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,076 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,076 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-02 03:54:09,077 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-02 03:54:09,077 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:09,077 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:09,077 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:09,079 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-02 03:54:09,079 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:09,079 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:09,079 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-02 03:54:09,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:09,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:09,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:09,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:09,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:09,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:09,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:09,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:09,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:09,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:54:09,081 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-02 03:54:09,081 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:09,081 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:09,081 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,081 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:09,083 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,083 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,083 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:09,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:09,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:09,083 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:09,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:09,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:09,084 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:09,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:09,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,085 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:09,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:09,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:09,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:09,087 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:09,087 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,087 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:09,087 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:09,087 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:09,087 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:09,087 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,087 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,087 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,087 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:09,087 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-02 03:54:09,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-02 03:54:09,087 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:09,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:09,088 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:09,088 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:09,088 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:09,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:09,188 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:09,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:09,192 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:09,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:09,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,194 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-02 03:54:09,194 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:09,194 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-02 03:54:09,194 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-02 03:54:09,194 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,195 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,196 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:09,196 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:09,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:09,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:09,196 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:09,196 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:09,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,197 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:09,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,197 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,197 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-02 03:54:09,197 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:09,197 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-02 03:54:09,197 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:09,197 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-02 03:54:09,197 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:09,198 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:09,198 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:09,198 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. started... 2023-11-02 03:54:09,198 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cb0f3da5a940148bb91c8b121c586d73 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:09,611 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205bb625b44ccfb4ae49badf3084d8943c7 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205bb625b44ccfb4ae49badf3084d8943c7 2023-11-02 03:54:10,016 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/72824b4399de486a820fa26b6afb01c4 2023-11-02 03:54:10,021 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/72824b4399de486a820fa26b6afb01c4 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/72824b4399de486a820fa26b6afb01c4 2023-11-02 03:54:10,025 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/72824b4399de486a820fa26b6afb01c4, entries=1, sequenceid=17, filesize=4.9 K 2023-11-02 03:54:10,026 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb0f3da5a940148bb91c8b121c586d73 in 828ms, sequenceid=17, compaction requested=true 2023-11-02 03:54:10,026 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cb0f3da5a940148bb91c8b121c586d73: 2023-11-02 03:54:10,026 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:10,026 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:10,026 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:10,026 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,026 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-02 03:54:10,026 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-02 03:54:10,028 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,029 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:10,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:10,029 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,029 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:10,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:10,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:10,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:10,030 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-02 03:54:10,030 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62e3efdb[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:10,030 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-02 03:54:10,030 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,031 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,032 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-02 03:54:10,032 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-02 03:54:10,032 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:10,032 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,032 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:10,033 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,033 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-02 03:54:10,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:10,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:10,033 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:10,033 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-02 03:54:10,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:10,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:10,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:10,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:10,035 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:54:10,035 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-02 03:54:10,035 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:10,035 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:10,036 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,036 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:10,037 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,037 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,037 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:10,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:10,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:10,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:10,038 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:10,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:10,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:10,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,039 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:10,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,042 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,043 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,043 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,043 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,043 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,043 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,043 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:10,043 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:10,043 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-02 03:54:10,043 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-02 03:54:10,043 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:10,043 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:10,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:10,043 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:10,043 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:10,043 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:10,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:10,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:10,143 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:10,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:10,148 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:10,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:10,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,150 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-02 03:54:10,150 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:10,150 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-02 03:54:10,150 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-02 03:54:10,150 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,151 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,152 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,152 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:10,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:10,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:10,152 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,152 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:10,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,153 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:10,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,153 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,153 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-02 03:54:10,153 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,153 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-02 03:54:10,153 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:10,153 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-02 03:54:10,154 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:10,154 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:10,154 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:10,154 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. started... 2023-11-02 03:54:10,154 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cb0f3da5a940148bb91c8b121c586d73 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:10,568 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015122894db6b8d7ad349069525122f3a4961da to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122894db6b8d7ad349069525122f3a4961da 2023-11-02 03:54:10,974 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/72165dfe605b475b91ef606cc8fd10a0 2023-11-02 03:54:10,979 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/72165dfe605b475b91ef606cc8fd10a0 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/72165dfe605b475b91ef606cc8fd10a0 2023-11-02 03:54:10,983 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/72165dfe605b475b91ef606cc8fd10a0, entries=1, sequenceid=21, filesize=4.9 K 2023-11-02 03:54:10,984 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb0f3da5a940148bb91c8b121c586d73 in 830ms, sequenceid=21, compaction requested=true 2023-11-02 03:54:10,984 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cb0f3da5a940148bb91c8b121c586d73: 2023-11-02 03:54:10,984 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:10,984 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:10,984 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:10,984 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,984 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-02 03:54:10,984 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-02 03:54:10,986 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,986 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:10,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:10,986 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,986 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:10,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:10,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:10,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:10,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-02 03:54:10,988 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@bad082b[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:10,988 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-02 03:54:10,988 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,989 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,989 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-02 03:54:10,989 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-02 03:54:10,989 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:10,989 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,989 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:10,991 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-02 03:54:10,991 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:10,991 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,991 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-02 03:54:10,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:10,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:10,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:10,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:10,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:10,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:10,994 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:54:10,994 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-02 03:54:10,994 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:10,994 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:10,994 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,994 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:10,998 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,998 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:10,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:10,998 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:10,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:10,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:10,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:10,999 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:10,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:10,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:11,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,000 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:11,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,002 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,002 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,002 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,002 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,002 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:11,002 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:11,003 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-02 03:54:11,003 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:11,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:11,003 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-02 03:54:11,003 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,003 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:11,002 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:11,003 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,003 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:11,003 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:11,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:11,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:11,103 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:11,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:11,108 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:11,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:11,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,109 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-02 03:54:11,109 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:11,109 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-02 03:54:11,109 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-02 03:54:11,110 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,110 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,111 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:11,111 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:11,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:11,111 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,111 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:11,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,112 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:11,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,112 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,113 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-02 03:54:11,113 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,113 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-02 03:54:11,113 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:11,113 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-02 03:54:11,113 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:11,113 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:11,113 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:11,113 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. started... 2023-11-02 03:54:11,113 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cb0f3da5a940148bb91c8b121c586d73 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:11,528 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123121b6315a552948d89b33e7608d4a1b3b to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015123121b6315a552948d89b33e7608d4a1b3b 2023-11-02 03:54:11,934 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/93b6cc2d0f054c29a5baf875a6cb41c7 2023-11-02 03:54:11,938 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/93b6cc2d0f054c29a5baf875a6cb41c7 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/93b6cc2d0f054c29a5baf875a6cb41c7 2023-11-02 03:54:11,942 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/93b6cc2d0f054c29a5baf875a6cb41c7, entries=1, sequenceid=25, filesize=4.9 K 2023-11-02 03:54:11,943 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb0f3da5a940148bb91c8b121c586d73 in 830ms, sequenceid=25, compaction requested=true 2023-11-02 03:54:11,943 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cb0f3da5a940148bb91c8b121c586d73: 2023-11-02 03:54:11,943 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:11,943 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:11,943 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:11,944 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,944 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-02 03:54:11,944 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-02 03:54:11,946 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,946 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:11,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:11,947 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,947 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:11,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:11,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:11,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:11,948 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-02 03:54:11,948 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3a45aaa4[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:11,948 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-02 03:54:11,948 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,949 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,949 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-02 03:54:11,949 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-02 03:54:11,949 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:11,949 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,949 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:11,951 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,951 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-02 03:54:11,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,951 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:11,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:11,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:11,951 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-02 03:54:11,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:11,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:11,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:11,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,953 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:11,953 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:54:11,953 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-02 03:54:11,953 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:11,953 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:11,953 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,953 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:11,955 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,955 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:11,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:11,955 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:11,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:11,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:11,955 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:11,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:11,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,956 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:11,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,960 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,960 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:11,960 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:11,960 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:11,960 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-02 03:54:11,960 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:11,960 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-02 03:54:11,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:11,960 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:11,960 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,960 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:11,960 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,960 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,960 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:11,960 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:11,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:11,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:11,960 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:12,061 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:12,062 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:12,065 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:12,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:12,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,067 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-02 03:54:12,067 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:12,067 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-02 03:54:12,067 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-02 03:54:12,068 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,068 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,069 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:12,069 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:12,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:12,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:12,069 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:12,069 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:12,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,070 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:12,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,070 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,070 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-02 03:54:12,070 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,070 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-02 03:54:12,070 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:12,070 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-02 03:54:12,071 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:12,071 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:12,071 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:12,071 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. started... 2023-11-02 03:54:12,071 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cb0f3da5a940148bb91c8b121c586d73 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:12,483 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601019a9ccbbca6ab44f3a8d2f1a6eef80385 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601019a9ccbbca6ab44f3a8d2f1a6eef80385 2023-11-02 03:54:12,888 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/809ff6b2f3734206a278b234e1b44547 2023-11-02 03:54:12,893 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/809ff6b2f3734206a278b234e1b44547 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/809ff6b2f3734206a278b234e1b44547 2023-11-02 03:54:12,897 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/809ff6b2f3734206a278b234e1b44547, entries=1, sequenceid=29, filesize=4.9 K 2023-11-02 03:54:12,898 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb0f3da5a940148bb91c8b121c586d73 in 827ms, sequenceid=29, compaction requested=true 2023-11-02 03:54:12,898 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cb0f3da5a940148bb91c8b121c586d73: 2023-11-02 03:54:12,898 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:12,898 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:12,898 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:12,898 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:12,898 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-02 03:54:12,898 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-02 03:54:12,900 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:12,900 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:12,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:12,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:12,900 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,900 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:12,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:12,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:12,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:12,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:12,901 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-02 03:54:12,901 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e0d0f1[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:12,901 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-02 03:54:12,901 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,902 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,902 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,903 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-02 03:54:12,903 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-02 03:54:12,903 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:12,903 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:12,903 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:12,905 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-02 03:54:12,905 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:12,905 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:12,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:12,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:12,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:12,905 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-02 03:54:12,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:12,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:12,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:12,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:12,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:12,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:12,908 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:54:12,908 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-02 03:54:12,908 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:12,908 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:12,908 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,908 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:12,909 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,909 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,910 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:12,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:12,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:12,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:12,910 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:12,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:12,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:12,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:12,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:12,911 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:12,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:12,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:12,913 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:12,914 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,914 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:12,914 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:12,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:12,914 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-02 03:54:12,914 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:12,914 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:12,914 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:12,914 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:12,914 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:12,914 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:12,914 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:12,914 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:12,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:12,914 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,014 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:13,015 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:13,019 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:13,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:13,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,020 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-02 03:54:13,020 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:13,021 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-02 03:54:13,021 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-02 03:54:13,021 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,021 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,022 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:13,022 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:13,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:13,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:13,023 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:13,023 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:13,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,023 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:13,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,023 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,023 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-02 03:54:13,024 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,024 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-02 03:54:13,024 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:13,024 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-02 03:54:13,024 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:13,024 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:13,024 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:13,024 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. started... 2023-11-02 03:54:13,024 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cb0f3da5a940148bb91c8b121c586d73 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:13,436 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103bb88886e699949dfae102f0c989af819 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103bb88886e699949dfae102f0c989af819 2023-11-02 03:54:13,841 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/6082823379944d9f98ff64f29d6575b4 2023-11-02 03:54:13,846 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/6082823379944d9f98ff64f29d6575b4 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/6082823379944d9f98ff64f29d6575b4 2023-11-02 03:54:13,850 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/6082823379944d9f98ff64f29d6575b4, entries=1, sequenceid=33, filesize=4.9 K 2023-11-02 03:54:13,851 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb0f3da5a940148bb91c8b121c586d73 in 827ms, sequenceid=33, compaction requested=true 2023-11-02 03:54:13,851 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cb0f3da5a940148bb91c8b121c586d73: 2023-11-02 03:54:13,851 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:13,851 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:13,851 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:13,851 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:13,851 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-02 03:54:13,851 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-02 03:54:13,854 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:13,854 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:13,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:13,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:13,854 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,854 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:13,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:13,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:13,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:13,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:13,855 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-02 03:54:13,856 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a807ae1[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:13,856 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-02 03:54:13,856 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,859 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,859 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-02 03:54:13,860 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-02 03:54:13,860 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:13,860 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:13,860 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:13,861 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:13,861 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-02 03:54:13,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:13,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:13,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:13,861 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:13,861 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-02 03:54:13,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:13,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:13,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:13,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:13,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:13,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:13,863 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:54:13,863 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-02 03:54:13,863 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:13,863 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:13,863 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,863 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:13,865 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,865 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,865 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:13,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:13,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:13,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:13,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:13,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:13,866 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:13,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:13,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,867 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:13,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:13,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:13,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:13,870 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:13,870 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,870 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,870 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:13,870 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,870 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:13,870 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-02 03:54:13,870 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:13,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:13,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:13,870 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-02 03:54:13,870 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:13,870 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:13,871 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:13,870 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,870 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:13,871 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:13,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:13,971 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:13,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:13,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:13,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:13,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,977 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-02 03:54:13,977 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:13,977 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-02 03:54:13,977 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-02 03:54:13,978 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,978 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,979 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:13,979 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:13,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:13,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:13,979 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:13,979 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:13,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,980 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:13,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,980 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,980 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-02 03:54:13,980 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:13,980 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-02 03:54:13,980 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:13,980 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-02 03:54:13,981 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:13,981 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:13,981 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:13,981 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. started... 2023-11-02 03:54:13,981 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cb0f3da5a940148bb91c8b121c586d73 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:14,394 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090714b8937218454a3db528c7b2efeacde1 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015090714b8937218454a3db528c7b2efeacde1 2023-11-02 03:54:14,799 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/a94b686089bd4f098ba9d2883381ea43 2023-11-02 03:54:14,804 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/a94b686089bd4f098ba9d2883381ea43 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/a94b686089bd4f098ba9d2883381ea43 2023-11-02 03:54:14,808 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/a94b686089bd4f098ba9d2883381ea43, entries=1, sequenceid=37, filesize=4.9 K 2023-11-02 03:54:14,808 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb0f3da5a940148bb91c8b121c586d73 in 827ms, sequenceid=37, compaction requested=true 2023-11-02 03:54:14,808 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cb0f3da5a940148bb91c8b121c586d73: 2023-11-02 03:54:14,808 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:14,809 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:14,809 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:14,809 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:14,809 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-02 03:54:14,809 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-02 03:54:14,810 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:14,810 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:14,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:14,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:14,811 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,811 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:14,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:14,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:14,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:14,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:14,812 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-02 03:54:14,812 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@198fd9ca[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:14,812 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-02 03:54:14,812 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,813 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,813 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-02 03:54:14,813 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-02 03:54:14,813 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:14,814 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:14,814 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:14,816 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-02 03:54:14,816 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:14,816 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:14,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:14,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:14,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:14,816 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-02 03:54:14,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:14,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:14,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:14,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:14,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:14,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:14,818 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:54:14,818 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-02 03:54:14,818 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:14,818 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:14,818 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,818 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:14,820 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,820 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:14,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,820 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:14,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:14,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:14,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:14,821 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:14,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:14,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:14,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,822 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:14,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:14,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:14,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:14,824 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:14,824 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:14,824 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:14,824 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,824 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-02 03:54:14,824 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:14,824 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:14,824 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-11-02 03:54:14,824 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:14,824 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:14,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:14,824 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:14,824 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,825 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:14,825 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,825 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:14,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:14,925 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:14,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:14,929 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:14,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:14,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,931 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-11-02 03:54:14,931 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:14,931 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-11-02 03:54:14,931 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-11-02 03:54:14,931 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,932 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,933 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:14,933 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:14,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:14,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:14,933 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:14,933 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:14,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,934 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:14,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,934 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,934 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-11-02 03:54:14,934 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:14,934 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-11-02 03:54:14,934 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:14,934 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-11-02 03:54:14,934 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:14,935 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:14,935 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:14,935 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. started... 2023-11-02 03:54:14,935 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing cb0f3da5a940148bb91c8b121c586d73 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:15,347 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120275e2cd847314c4caa4fb15a860e305c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151120275e2cd847314c4caa4fb15a860e305c 2023-11-02 03:54:15,753 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/5517f192e0fa41dcafc8a9675697e0cd 2023-11-02 03:54:15,758 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/5517f192e0fa41dcafc8a9675697e0cd as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/5517f192e0fa41dcafc8a9675697e0cd 2023-11-02 03:54:15,762 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/5517f192e0fa41dcafc8a9675697e0cd, entries=1, sequenceid=41, filesize=4.9 K 2023-11-02 03:54:15,763 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for cb0f3da5a940148bb91c8b121c586d73 in 828ms, sequenceid=41, compaction requested=true 2023-11-02 03:54:15,763 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for cb0f3da5a940148bb91c8b121c586d73: 2023-11-02 03:54:15,763 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:15,763 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:15,763 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:15,763 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:15,763 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-11-02 03:54:15,763 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-11-02 03:54:15,765 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,765 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:15,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:15,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:15,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:15,765 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,765 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:15,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:15,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:15,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:15,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:15,767 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-11-02 03:54:15,767 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e8840d5[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:15,767 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-11-02 03:54:15,767 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,768 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,768 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-11-02 03:54:15,768 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-11-02 03:54:15,768 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:15,768 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:15,768 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:15,770 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:15,770 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-11-02 03:54:15,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:15,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:15,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:15,770 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:15,770 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-11-02 03:54:15,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:15,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:15,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:15,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:15,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:15,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:15,772 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:54:15,772 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-11-02 03:54:15,772 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:15,772 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:15,772 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,772 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:15,775 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,775 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:15,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:15,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,775 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:15,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:15,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:15,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:15,775 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:15,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:15,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:15,777 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:15,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:15,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:15,782 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:15,782 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:15,782 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:15,782 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:15,782 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:15,782 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-11-02 03:54:15,783 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:15,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:15,782 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,783 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:15,783 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,783 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,783 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,783 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:15,783 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:15,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:15,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:15,883 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:15,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:15,885 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41275] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-11-02 03:54:15,887 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] procedure2.ProcedureExecutor(1028): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-02 03:54:15,887 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-02 03:54:15,888 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:54:15,888 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698897855888 2023-11-02 03:54:15,891 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-02 03:54:15,892 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-11-02 03:54:15,892 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-02 03:54:15,893 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] zookeeper.ReadOnlyZKClient(139): Connect 0x1e097ec5 to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:54:15,897 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@782adf3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:54:15,897 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-02 03:54:16,738 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103fdf5c46093544aac881abe5c80df6007 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103fdf5c46093544aac881abe5c80df6007 2023-11-02 03:54:16,739 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-02 03:54:16,756 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-02 03:54:16,757 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56030, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-02 03:54:16,763 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-02 03:54:16,764 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50622, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-02 03:54:16,779 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0389b26504ec45c7820a9221e280ad87 first=Optional[r3] last=Optional[r6] 2023-11-02 03:54:16,780 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0389b26504ec45c7820a9221e280ad87}] 2023-11-02 03:54:16,783 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0389b26504ec45c7820a9221e280ad87 for inclusion in cb0f3da5a940148bb91c8b121c586d73/family1 2023-11-02 03:54:16,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-11-02 03:54:16,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last= 2023-11-02 03:54:16,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for cb0f3da5a940148bb91c8b121c586d73: 2023-11-02 03:54:16,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0389b26504ec45c7820a9221e280ad87 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__5ej4logcmqrrfdg1elifk34duedao7dsi3cl90178pundpictvsqu65lmcfb7tb8/family1/0389b26504ec45c7820a9221e280ad87 2023-11-02 03:54:16,788 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__5ej4logcmqrrfdg1elifk34duedao7dsi3cl90178pundpictvsqu65lmcfb7tb8/family1/0389b26504ec45c7820a9221e280ad87 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/e0b606467ee14a33a1594e9280fde413_SeqId_44_ 2023-11-02 03:54:16,789 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__5ej4logcmqrrfdg1elifk34duedao7dsi3cl90178pundpictvsqu65lmcfb7tb8/family1/0389b26504ec45c7820a9221e280ad87 into cb0f3da5a940148bb91c8b121c586d73/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/e0b606467ee14a33a1594e9280fde413_SeqId_44_ - updating store file list. 2023-11-02 03:54:16,792 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/e0b606467ee14a33a1594e9280fde413_SeqId_44_ into cb0f3da5a940148bb91c8b121c586d73/family1 2023-11-02 03:54:16,792 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__5ej4logcmqrrfdg1elifk34duedao7dsi3cl90178pundpictvsqu65lmcfb7tb8/family1/0389b26504ec45c7820a9221e280ad87 into cb0f3da5a940148bb91c8b121c586d73/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/e0b606467ee14a33a1594e9280fde413_SeqId_44_) 2023-11-02 03:54:16,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__5ej4logcmqrrfdg1elifk34duedao7dsi3cl90178pundpictvsqu65lmcfb7tb8/family1/0389b26504ec45c7820a9221e280ad87 2023-11-02 03:54:16,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-02 03:54:16,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-02 03:54:16,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(1912): cb0f3da5a940148bb91c8b121c586d73/family1 is initiating minor compaction 2023-11-02 03:54:16,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7228cbd; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:54:16,796 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of cb0f3da5a940148bb91c8b121c586d73/family1 in testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:16,796 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/211b43445f4a4b08be6383c73d0f2794, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/97931e699be44669b8acdeadf644b7c0, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/72824b4399de486a820fa26b6afb01c4, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/72165dfe605b475b91ef606cc8fd10a0, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/93b6cc2d0f054c29a5baf875a6cb41c7, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/809ff6b2f3734206a278b234e1b44547, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/6082823379944d9f98ff64f29d6575b4, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/a94b686089bd4f098ba9d2883381ea43, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/5517f192e0fa41dcafc8a9675697e0cd, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/e0b606467ee14a33a1594e9280fde413_SeqId_44_] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp, totalSize=50.0 K 2023-11-02 03:54:16,796 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 211b43445f4a4b08be6383c73d0f2794, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-02 03:54:16,796 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 97931e699be44669b8acdeadf644b7c0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-02 03:54:16,796 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 72824b4399de486a820fa26b6afb01c4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-02 03:54:16,796 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 72165dfe605b475b91ef606cc8fd10a0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-02 03:54:16,796 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 93b6cc2d0f054c29a5baf875a6cb41c7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-02 03:54:16,796 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 809ff6b2f3734206a278b234e1b44547, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-02 03:54:16,796 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 6082823379944d9f98ff64f29d6575b4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-02 03:54:16,796 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting a94b686089bd4f098ba9d2883381ea43, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-02 03:54:16,796 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 5517f192e0fa41dcafc8a9675697e0cd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-02 03:54:16,796 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting e0b606467ee14a33a1594e9280fde413_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-02 03:54:16,797 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-02 03:54:16,798 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:54:16,800 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122894db6b8d7ad349069525122f3a4961da to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015122894db6b8d7ad349069525122f3a4961da 2023-11-02 03:54:16,801 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015123121b6315a552948d89b33e7608d4a1b3b to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015123121b6315a552948d89b33e7608d4a1b3b 2023-11-02 03:54:16,802 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601019a9ccbbca6ab44f3a8d2f1a6eef80385 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601019a9ccbbca6ab44f3a8d2f1a6eef80385 2023-11-02 03:54:16,803 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103bb88886e699949dfae102f0c989af819 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103bb88886e699949dfae102f0c989af819 2023-11-02 03:54:16,803 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-11-02 03:54:16,803 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-11-02 03:54:16,804 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-02 03:54:16,804 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-02 03:54:16,804 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-02 03:54:16,804 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-02 03:54:16,804 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-02 03:54:16,804 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-02 03:54:16,804 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-02 03:54:16,804 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-02 03:54:16,831 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] throttle.PressureAwareThroughputController(145): cb0f3da5a940148bb91c8b121c586d73#family1#compaction#48 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:54:17,270 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/5d957829b92e4134bba9b74c5fe5c2cc as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/5d957829b92e4134bba9b74c5fe5c2cc 2023-11-02 03:54:17,274 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205e1a90a2ae1d34853931e696aab6fdad8 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205e1a90a2ae1d34853931e696aab6fdad8 2023-11-02 03:54:17,275 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-02 03:54:17,275 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in cb0f3da5a940148bb91c8b121c586d73/family1 of cb0f3da5a940148bb91c8b121c586d73 into 5d957829b92e4134bba9b74c5fe5c2cc(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-02 03:54:17,275 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for cb0f3da5a940148bb91c8b121c586d73: 2023-11-02 03:54:17,275 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1698897256796; duration=0sec 2023-11-02 03:54:17,275 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:54:17,299 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7d58e53a31a74fc79904e7864fad9a10 first=Optional[r00] last=Optional[r2] 2023-11-02 03:54:17,300 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7d58e53a31a74fc79904e7864fad9a10}] 2023-11-02 03:54:17,301 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7d58e53a31a74fc79904e7864fad9a10 for inclusion in cb0f3da5a940148bb91c8b121c586d73/family1 2023-11-02 03:54:17,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-11-02 03:54:17,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last= 2023-11-02 03:54:17,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for cb0f3da5a940148bb91c8b121c586d73: 2023-11-02 03:54:17,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7d58e53a31a74fc79904e7864fad9a10 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__avq45l2s9h7f8o0rq1r7rgdjn71mgnouor9m4is88ihmv973pk4ne0a00nhgg7t7/family1/7d58e53a31a74fc79904e7864fad9a10 2023-11-02 03:54:17,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__avq45l2s9h7f8o0rq1r7rgdjn71mgnouor9m4is88ihmv973pk4ne0a00nhgg7t7/family1/7d58e53a31a74fc79904e7864fad9a10 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/b5cd9e2aa4034cbba5eb982538ee0ead_SeqId_47_ 2023-11-02 03:54:17,307 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__avq45l2s9h7f8o0rq1r7rgdjn71mgnouor9m4is88ihmv973pk4ne0a00nhgg7t7/family1/7d58e53a31a74fc79904e7864fad9a10 into cb0f3da5a940148bb91c8b121c586d73/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/b5cd9e2aa4034cbba5eb982538ee0ead_SeqId_47_ - updating store file list. 2023-11-02 03:54:17,311 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/b5cd9e2aa4034cbba5eb982538ee0ead_SeqId_47_ into cb0f3da5a940148bb91c8b121c586d73/family1 2023-11-02 03:54:17,311 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__avq45l2s9h7f8o0rq1r7rgdjn71mgnouor9m4is88ihmv973pk4ne0a00nhgg7t7/family1/7d58e53a31a74fc79904e7864fad9a10 into cb0f3da5a940148bb91c8b121c586d73/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/b5cd9e2aa4034cbba5eb982538ee0ead_SeqId_47_) 2023-11-02 03:54:17,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyPolicy__avq45l2s9h7f8o0rq1r7rgdjn71mgnouor9m4is88ihmv973pk4ne0a00nhgg7t7/family1/7d58e53a31a74fc79904e7864fad9a10 2023-11-02 03:54:17,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-02 03:54:17,312 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 345120 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-02 03:54:17,312 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(1912): cb0f3da5a940148bb91c8b121c586d73/family1 is initiating minor compaction (all files) 2023-11-02 03:54:17,312 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4398b591; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:54:17,312 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of cb0f3da5a940148bb91c8b121c586d73/family1 in testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:54:17,312 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/618237a641134f49be947204037abfca, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/5d957829b92e4134bba9b74c5fe5c2cc, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/b5cd9e2aa4034cbba5eb982538ee0ead_SeqId_47_] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp, totalSize=337.0 K 2023-11-02 03:54:17,313 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 618237a641134f49be947204037abfca, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878824741 2023-11-02 03:54:17,313 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 5d957829b92e4134bba9b74c5fe5c2cc, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621224741 2023-11-02 03:54:17,314 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting b5cd9e2aa4034cbba5eb982538ee0ead_SeqId_47_, keycount=1002, bloomtype=ROW, size=165.4 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878824741 2023-11-02 03:54:17,314 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-02 03:54:17,314 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:54:17,316 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130b45d62b653d641089cde138a64234e08 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130b45d62b653d641089cde138a64234e08 2023-11-02 03:54:17,317 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120149882fcc3f8a49419c065a0c9bff2f0a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120149882fcc3f8a49419c065a0c9bff2f0a 2023-11-02 03:54:17,319 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205bb625b44ccfb4ae49badf3084d8943c7 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205bb625b44ccfb4ae49badf3084d8943c7 2023-11-02 03:54:17,319 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-11-02 03:54:17,319 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-11-02 03:54:17,319 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-11-02 03:54:17,319 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-11-02 03:54:17,319 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:54:17,320 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-11-02 03:54:17,322 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.4340 sec 2023-11-02 03:54:17,329 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] throttle.PressureAwareThroughputController(145): cb0f3da5a940148bb91c8b121c586d73#family1#compaction#49 average throughput is 22.09 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:54:17,362 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/211b43445f4a4b08be6383c73d0f2794, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/97931e699be44669b8acdeadf644b7c0, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/72824b4399de486a820fa26b6afb01c4, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/72165dfe605b475b91ef606cc8fd10a0, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/93b6cc2d0f054c29a5baf875a6cb41c7, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/809ff6b2f3734206a278b234e1b44547, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/6082823379944d9f98ff64f29d6575b4, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/a94b686089bd4f098ba9d2883381ea43, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/5517f192e0fa41dcafc8a9675697e0cd, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/e0b606467ee14a33a1594e9280fde413_SeqId_44_] to archive 2023-11-02 03:54:17,363 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:54:17,365 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/211b43445f4a4b08be6383c73d0f2794 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/211b43445f4a4b08be6383c73d0f2794 2023-11-02 03:54:17,367 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/97931e699be44669b8acdeadf644b7c0 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/97931e699be44669b8acdeadf644b7c0 2023-11-02 03:54:17,369 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/72824b4399de486a820fa26b6afb01c4 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/72824b4399de486a820fa26b6afb01c4 2023-11-02 03:54:17,370 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/72165dfe605b475b91ef606cc8fd10a0 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/72165dfe605b475b91ef606cc8fd10a0 2023-11-02 03:54:17,372 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/93b6cc2d0f054c29a5baf875a6cb41c7 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/93b6cc2d0f054c29a5baf875a6cb41c7 2023-11-02 03:54:17,373 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/809ff6b2f3734206a278b234e1b44547 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/809ff6b2f3734206a278b234e1b44547 2023-11-02 03:54:17,375 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/6082823379944d9f98ff64f29d6575b4 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/6082823379944d9f98ff64f29d6575b4 2023-11-02 03:54:17,376 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/a94b686089bd4f098ba9d2883381ea43 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/a94b686089bd4f098ba9d2883381ea43 2023-11-02 03:54:17,377 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/5517f192e0fa41dcafc8a9675697e0cd to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/5517f192e0fa41dcafc8a9675697e0cd 2023-11-02 03:54:17,378 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/e0b606467ee14a33a1594e9280fde413_SeqId_44_ to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/e0b606467ee14a33a1594e9280fde413_SeqId_44_ 2023-11-02 03:54:17,379 DEBUG [hfile_cleaner-dir-scan-pool-1] cleaner.CleanerChore(483): Couldn't delete 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default' yet because it isn't empty w/exception. org.apache.hadoop.fs.PathIsNotEmptyDirectoryException: ``/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) ': Directory is not empty: `/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1595) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:881) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:878) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:878) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$4(CleanerChore.java:453) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:478) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:453) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:61) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:439) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$2(CleanerChore.java:432) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): `/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.delete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:572) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1593) ... 35 more 2023-11-02 03:54:17,438 INFO [Listener at localhost/43647] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=264 (was 260) - Thread LEAK? -, OpenFileDescriptor=656 (was 642) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=225 (was 266), ProcessCount=170 (was 170), AvailableMemoryMB=11224 (was 11270) 2023-11-02 03:54:17,450 INFO [Listener at localhost/43647] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=264, OpenFileDescriptor=656, MaxFileDescriptor=60000, SystemLoadAverage=225, ProcessCount=170, AvailableMemoryMB=11222 2023-11-02 03:54:17,452 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-02 03:54:17,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:17,454 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-11-02 03:54:17,454 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-11-02 03:54:17,455 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-02 03:54:17,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-02 03:54:17,456 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:17,457 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e empty. 2023-11-02 03:54:17,457 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:17,457 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-11-02 03:54:17,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-02 03:54:17,757 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-02 03:54:17,865 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-11-02 03:54:17,866 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7675): creating {ENCODED => 06d225f5eeba431dae3475159f87d35e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp 2023-11-02 03:54:18,058 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-02 03:54:18,272 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:18,272 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1604): Closing 06d225f5eeba431dae3475159f87d35e, disabling compactions & flushes 2023-11-02 03:54:18,272 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:18,272 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:18,272 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. after waiting 0 ms 2023-11-02 03:54:18,272 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:18,272 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:18,272 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1558): Region close journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:54:18,274 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-11-02 03:54:18,275 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698897258275"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897258275"}]},"ts":"1698897258275"} 2023-11-02 03:54:18,277 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-02 03:54:18,278 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-02 03:54:18,278 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897258278"}]},"ts":"1698897258278"} 2023-11-02 03:54:18,279 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-11-02 03:54:18,283 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=06d225f5eeba431dae3475159f87d35e, ASSIGN}] 2023-11-02 03:54:18,285 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=06d225f5eeba431dae3475159f87d35e, ASSIGN 2023-11-02 03:54:18,285 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=06d225f5eeba431dae3475159f87d35e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=false 2023-11-02 03:54:18,436 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=06d225f5eeba431dae3475159f87d35e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:18,437 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698897258436"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897258436"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897258436"}]},"ts":"1698897258436"} 2023-11-02 03:54:18,438 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure 06d225f5eeba431dae3475159f87d35e, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:54:18,554 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/.tmp/family1/387b2291d4d14c4c981bae2b1c0765a4 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/387b2291d4d14c4c981bae2b1c0765a4 2023-11-02 03:54:18,559 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in cb0f3da5a940148bb91c8b121c586d73/family1 of cb0f3da5a940148bb91c8b121c586d73 into 387b2291d4d14c4c981bae2b1c0765a4(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-02 03:54:18,559 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for cb0f3da5a940148bb91c8b121c586d73: 2023-11-02 03:54:18,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-02 03:54:18,559 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73., storeName=family1, fileCount=3, fileSize=337.0 K (165.1 K, 6.6 K, 165.4 K), priority=2, time=1698897257312; duration=1sec 2023-11-02 03:54:18,559 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:54:18,590 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:18,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:18,592 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 06d225f5eeba431dae3475159f87d35e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.', STARTKEY => '', ENDKEY => ''} 2023-11-02 03:54:18,592 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:18,593 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:18,593 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:18,593 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:18,594 INFO [StoreOpener-06d225f5eeba431dae3475159f87d35e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:18,595 DEBUG [StoreOpener-06d225f5eeba431dae3475159f87d35e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1 2023-11-02 03:54:18,595 DEBUG [StoreOpener-06d225f5eeba431dae3475159f87d35e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1 2023-11-02 03:54:18,595 INFO [StoreOpener-06d225f5eeba431dae3475159f87d35e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 06d225f5eeba431dae3475159f87d35e columnFamilyName family1 2023-11-02 03:54:18,596 INFO [StoreOpener-06d225f5eeba431dae3475159f87d35e-1] regionserver.HStore(310): Store=06d225f5eeba431dae3475159f87d35e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:54:18,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:18,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:18,600 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:18,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:54:18,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 06d225f5eeba431dae3475159f87d35e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60645411, jitterRate=-0.09631295502185822}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-02 03:54:18,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:54:18,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e., pid=41, masterSystemTime=1698897258590 2023-11-02 03:54:18,604 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:18,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:18,604 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=06d225f5eeba431dae3475159f87d35e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:18,604 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698897258604"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897258604"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897258604"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897258604"}]},"ts":"1698897258604"} 2023-11-02 03:54:18,607 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=41, resume processing ppid=40 2023-11-02 03:54:18,607 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure 06d225f5eeba431dae3475159f87d35e, server=jenkins-hbase4.apache.org,36627,1698897201574 in 168 msec 2023-11-02 03:54:18,608 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=40, resume processing ppid=39 2023-11-02 03:54:18,608 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=06d225f5eeba431dae3475159f87d35e, ASSIGN in 324 msec 2023-11-02 03:54:18,609 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-02 03:54:18,609 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897258609"}]},"ts":"1698897258609"} 2023-11-02 03:54:18,610 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-11-02 03:54:18,612 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-11-02 03:54:18,613 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1600 sec 2023-11-02 03:54:18,653 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/618237a641134f49be947204037abfca, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/5d957829b92e4134bba9b74c5fe5c2cc, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/b5cd9e2aa4034cbba5eb982538ee0ead_SeqId_47_] to archive 2023-11-02 03:54:18,654 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:54:18,655 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/618237a641134f49be947204037abfca to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/618237a641134f49be947204037abfca 2023-11-02 03:54:18,657 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/5d957829b92e4134bba9b74c5fe5c2cc to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/5d957829b92e4134bba9b74c5fe5c2cc 2023-11-02 03:54:18,658 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/b5cd9e2aa4034cbba5eb982538ee0ead_SeqId_47_ to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/family1/b5cd9e2aa4034cbba5eb982538ee0ead_SeqId_47_ 2023-11-02 03:54:19,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-11-02 03:54:19,560 INFO [Listener at localhost/43647] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-11-02 03:54:19,580 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:19,582 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:19,582 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-02 03:54:19,582 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:19,582 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-02 03:54:19,582 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-02 03:54:19,582 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:19,582 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:19,584 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:19,584 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:19,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:19,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:19,584 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:19,584 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:19,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:19,584 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:19,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:19,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:19,585 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:19,585 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:19,585 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-02 03:54:19,585 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:19,585 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-02 03:54:19,585 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:19,585 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:19,585 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:19,585 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. started... 2023-11-02 03:54:19,586 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 06d225f5eeba431dae3475159f87d35e 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-02 03:54:20,003 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130158e1b895ea04067bd813d734a099d12 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130158e1b895ea04067bd813d734a099d12 2023-11-02 03:54:20,407 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/05b186f8a71b4f2d9e9e71739bcd324f 2023-11-02 03:54:20,412 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/05b186f8a71b4f2d9e9e71739bcd324f as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/05b186f8a71b4f2d9e9e71739bcd324f 2023-11-02 03:54:20,416 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/05b186f8a71b4f2d9e9e71739bcd324f, entries=1000, sequenceid=5, filesize=191.4 K 2023-11-02 03:54:20,417 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 06d225f5eeba431dae3475159f87d35e in 831ms, sequenceid=5, compaction requested=false 2023-11-02 03:54:20,417 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-02 03:54:20,417 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:54:20,417 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:20,418 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:20,418 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:20,418 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:20,418 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-02 03:54:20,418 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-02 03:54:20,419 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:20,419 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:20,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:20,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:20,420 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,420 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:20,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:20,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:20,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:20,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:20,421 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-02 03:54:20,421 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@fee7ead[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:20,421 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-02 03:54:20,421 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,428 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,429 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-02 03:54:20,429 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-02 03:54:20,429 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:20,429 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:20,429 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:20,430 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:20,430 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-02 03:54:20,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:20,430 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:20,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:20,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:20,430 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-02 03:54:20,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:20,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:20,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:20,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:20,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:20,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:20,433 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-02 03:54:20,433 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-02 03:54:20,433 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:20,433 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:20,433 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,433 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:20,434 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,434 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,434 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:20,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:20,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:20,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,435 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:20,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:20,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:20,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:20,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:20,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,436 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:20,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:20,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:20,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:20,438 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:20,438 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,438 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,438 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:20,438 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:20,438 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-02 03:54:20,438 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:20,438 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,438 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-02 03:54:20,438 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:20,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:20,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:20,439 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:20,438 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:20,439 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:20,438 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,439 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:20,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:20,539 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:20,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:20,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:20,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:20,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,545 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-02 03:54:20,545 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:20,545 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-02 03:54:20,545 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-02 03:54:20,546 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,546 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,550 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:20,550 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:20,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:20,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:20,550 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:20,550 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:20,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,551 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:20,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,551 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,551 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-02 03:54:20,551 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:20,551 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-02 03:54:20,551 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:20,551 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-02 03:54:20,552 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:20,552 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:20,552 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:20,552 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. started... 2023-11-02 03:54:20,552 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 06d225f5eeba431dae3475159f87d35e 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-02 03:54:20,964 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511286f905254e253434d9e2ddbb831035e56 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511286f905254e253434d9e2ddbb831035e56 2023-11-02 03:54:21,369 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/a4bdf8329d5e4137986b41aa073a3806 2023-11-02 03:54:21,374 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/a4bdf8329d5e4137986b41aa073a3806 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/a4bdf8329d5e4137986b41aa073a3806 2023-11-02 03:54:21,379 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/a4bdf8329d5e4137986b41aa073a3806, entries=1, sequenceid=9, filesize=5.0 K 2023-11-02 03:54:21,380 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 06d225f5eeba431dae3475159f87d35e in 828ms, sequenceid=9, compaction requested=false 2023-11-02 03:54:21,380 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:54:21,380 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:21,380 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:21,380 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:21,380 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:21,380 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-02 03:54:21,380 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-02 03:54:21,382 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:21,382 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:21,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:21,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:21,383 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,383 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:21,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:21,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:21,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:21,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:21,384 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-02 03:54:21,384 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76405ac3[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:21,384 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-02 03:54:21,384 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,386 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,386 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-02 03:54:21,386 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-02 03:54:21,386 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:21,386 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:21,387 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:21,388 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-02 03:54:21,388 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:21,388 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:21,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:21,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:21,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:21,388 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-02 03:54:21,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:21,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:21,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:21,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:21,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:21,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:21,390 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-02 03:54:21,390 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-02 03:54:21,390 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:21,390 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:21,390 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,390 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:21,392 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,392 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:21,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,392 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:21,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:21,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:21,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:21,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:21,393 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:21,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:21,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:21,394 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:21,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:21,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:21,400 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:21,400 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:21,400 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,400 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:21,400 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:21,400 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-02 03:54:21,400 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:21,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:21,400 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,400 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:21,400 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:21,400 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,400 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,400 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,400 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:21,401 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:21,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:21,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:21,501 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:21,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:21,505 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:21,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:21,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,507 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-02 03:54:21,507 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:21,507 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-02 03:54:21,507 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-02 03:54:21,507 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,508 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,509 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:21,509 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:21,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:21,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:21,509 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:21,509 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:21,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,510 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:21,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,510 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,510 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-02 03:54:21,510 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:21,510 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-02 03:54:21,510 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:21,510 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-02 03:54:21,510 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:21,510 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:21,511 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:21,511 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. started... 2023-11-02 03:54:21,511 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 06d225f5eeba431dae3475159f87d35e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:21,922 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120187f35de2cdcc4313bedb673c83bc6db1 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120187f35de2cdcc4313bedb673c83bc6db1 2023-11-02 03:54:22,328 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/2420df720bf345c6a6779696b1c9aab5 2023-11-02 03:54:22,333 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/2420df720bf345c6a6779696b1c9aab5 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/2420df720bf345c6a6779696b1c9aab5 2023-11-02 03:54:22,337 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/2420df720bf345c6a6779696b1c9aab5, entries=1, sequenceid=13, filesize=5.0 K 2023-11-02 03:54:22,337 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 06d225f5eeba431dae3475159f87d35e in 826ms, sequenceid=13, compaction requested=true 2023-11-02 03:54:22,338 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:54:22,338 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:22,338 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:22,338 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:22,338 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,338 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-02 03:54:22,338 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-02 03:54:22,340 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,340 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:22,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:22,340 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,340 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:22,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:22,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:22,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:22,341 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-02 03:54:22,341 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@11668c48[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:22,341 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-02 03:54:22,341 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,343 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,343 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-02 03:54:22,343 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-02 03:54:22,343 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:22,343 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,343 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:22,345 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-02 03:54:22,345 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:22,345 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,345 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-02 03:54:22,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:22,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:22,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:22,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:22,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:22,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:22,348 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-02 03:54:22,348 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-02 03:54:22,348 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:22,348 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:22,348 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,348 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:22,349 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,349 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,349 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:22,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:22,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:22,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:22,350 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:22,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:22,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:22,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,351 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:22,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,353 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,353 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,354 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:22,354 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:22,354 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:22,354 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-02 03:54:22,354 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-02 03:54:22,354 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:22,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:22,354 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,354 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:22,354 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,354 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,354 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,354 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:22,354 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:22,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:22,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:22,454 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:22,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:22,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:22,461 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:22,461 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,461 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-02 03:54:22,461 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:22,461 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-02 03:54:22,461 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-02 03:54:22,461 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,461 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,463 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:22,463 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,463 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:22,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:22,463 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,463 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:22,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,464 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:22,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,464 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,464 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-02 03:54:22,464 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,464 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-02 03:54:22,464 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:22,465 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-02 03:54:22,465 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:22,465 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:22,465 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:22,465 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. started... 2023-11-02 03:54:22,465 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 06d225f5eeba431dae3475159f87d35e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:22,876 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205c0914d2f42b1422c89a8d09bdbf8d561 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205c0914d2f42b1422c89a8d09bdbf8d561 2023-11-02 03:54:22,881 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/ed26775f24b44d49acf7bd87b1ded286 2023-11-02 03:54:22,885 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/ed26775f24b44d49acf7bd87b1ded286 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/ed26775f24b44d49acf7bd87b1ded286 2023-11-02 03:54:22,891 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/ed26775f24b44d49acf7bd87b1ded286, entries=1, sequenceid=17, filesize=5.0 K 2023-11-02 03:54:22,891 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 06d225f5eeba431dae3475159f87d35e in 426ms, sequenceid=17, compaction requested=true 2023-11-02 03:54:22,892 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:54:22,892 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:22,892 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:22,892 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:22,892 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,892 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-02 03:54:22,892 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-02 03:54:22,894 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,894 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:22,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:22,894 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,894 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:22,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:22,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:22,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:22,895 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-02 03:54:22,895 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5da4f859[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:22,895 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-02 03:54:22,895 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,896 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,897 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-02 03:54:22,897 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-02 03:54:22,897 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:22,897 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,897 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:22,898 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-02 03:54:22,898 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,898 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:22,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:22,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:22,898 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-02 03:54:22,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:22,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:22,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:22,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:22,901 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-02 03:54:22,901 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-02 03:54:22,901 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:22,901 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:22,901 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,901 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:22,905 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,905 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:22,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:22,905 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:22,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:22,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:22,906 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:22,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:22,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,907 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:22,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,909 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,909 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:22,909 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:22,909 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-02 03:54:22,909 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:22,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:22,909 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:22,909 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,909 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:22,909 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:22,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:22,909 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:22,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:22,909 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:22,909 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-02 03:54:22,909 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,910 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,910 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:22,943 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.07692307692307693, tune throughput to 53.85 MB/second 2023-11-02 03:54:23,010 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:23,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:23,014 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:23,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:23,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,016 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-02 03:54:23,016 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:23,016 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-02 03:54:23,016 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-02 03:54:23,016 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,016 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,018 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:23,018 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:23,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:23,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:23,018 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:23,018 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:23,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,018 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:23,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,019 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,019 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-02 03:54:23,019 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,019 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-02 03:54:23,019 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:23,019 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-02 03:54:23,019 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:23,019 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:23,019 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:23,019 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. started... 2023-11-02 03:54:23,019 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 06d225f5eeba431dae3475159f87d35e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:23,432 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512287d66442bb2e243fcbcd97943b7017772 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512287d66442bb2e243fcbcd97943b7017772 2023-11-02 03:54:23,837 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/eb3bc0ade0f64e24acccb673678e302c 2023-11-02 03:54:23,842 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/eb3bc0ade0f64e24acccb673678e302c as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/eb3bc0ade0f64e24acccb673678e302c 2023-11-02 03:54:23,846 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/eb3bc0ade0f64e24acccb673678e302c, entries=1, sequenceid=21, filesize=5.0 K 2023-11-02 03:54:23,847 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 06d225f5eeba431dae3475159f87d35e in 828ms, sequenceid=21, compaction requested=true 2023-11-02 03:54:23,847 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:54:23,847 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:23,847 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:23,847 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:23,847 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:23,847 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-02 03:54:23,847 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-02 03:54:23,853 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,853 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:23,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:23,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:23,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:23,853 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,853 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:23,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:23,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:23,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:23,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:23,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-02 03:54:23,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@21c3ee56[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:23,854 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-02 03:54:23,854 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,855 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,855 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-02 03:54:23,855 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-02 03:54:23,856 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:23,856 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:23,856 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:23,857 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-02 03:54:23,857 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:23,857 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:23,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:23,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:23,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:23,857 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-02 03:54:23,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:23,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:23,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:23,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:23,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:23,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:23,859 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-02 03:54:23,859 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-02 03:54:23,859 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:23,859 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:23,859 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,859 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:23,861 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,861 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:23,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,861 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:23,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:23,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:23,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:23,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:23,862 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:23,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:23,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:23,863 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:23,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:23,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:23,866 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:23,866 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,866 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,866 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:23,866 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:23,866 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-02 03:54:23,867 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:23,866 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:23,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:23,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:23,866 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:23,866 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,867 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:23,867 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-02 03:54:23,867 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:23,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:23,867 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:23,967 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:23,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:23,972 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:23,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:23,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,973 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-02 03:54:23,973 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:23,973 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-02 03:54:23,973 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-02 03:54:23,974 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,974 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,975 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:23,975 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:23,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:23,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:23,975 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:23,975 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:23,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,976 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:23,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,976 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,976 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-02 03:54:23,976 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:23,976 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-02 03:54:23,976 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:23,977 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-02 03:54:23,977 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:23,977 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:23,977 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:23,977 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. started... 2023-11-02 03:54:23,977 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 06d225f5eeba431dae3475159f87d35e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:24,130 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-02 03:54:24,389 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231810fcbc36d9f4fc5b2adc455b46ebe2d to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231810fcbc36d9f4fc5b2adc455b46ebe2d 2023-11-02 03:54:24,795 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/d9f111deb459406aaa770aecf31c5f49 2023-11-02 03:54:24,800 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/d9f111deb459406aaa770aecf31c5f49 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/d9f111deb459406aaa770aecf31c5f49 2023-11-02 03:54:24,804 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/d9f111deb459406aaa770aecf31c5f49, entries=1, sequenceid=25, filesize=5.0 K 2023-11-02 03:54:24,805 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 06d225f5eeba431dae3475159f87d35e in 828ms, sequenceid=25, compaction requested=true 2023-11-02 03:54:24,805 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:54:24,805 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:24,805 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:24,805 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:24,805 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:24,805 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-02 03:54:24,805 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-02 03:54:24,807 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:24,807 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:24,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:24,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:24,807 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,807 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:24,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:24,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:24,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:24,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:24,808 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-02 03:54:24,808 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55a12eef[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:24,808 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-02 03:54:24,809 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,810 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,810 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-02 03:54:24,810 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-02 03:54:24,810 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:24,810 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:24,810 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:24,812 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-02 03:54:24,812 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:24,812 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:24,813 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-02 03:54:24,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:24,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:24,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:24,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:24,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:24,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:24,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:24,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:24,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:24,815 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-02 03:54:24,815 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-02 03:54:24,815 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:24,815 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:24,815 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,815 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:24,817 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,817 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:24,817 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:24,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:24,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:24,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:24,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:24,817 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:24,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:24,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:24,819 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:24,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:24,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:24,821 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:24,821 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,821 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:24,821 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,821 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-02 03:54:24,821 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:24,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:24,821 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:24,821 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:24,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:24,821 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:24,822 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:24,822 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:24,822 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,822 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:24,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:24,822 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,922 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:24,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:24,926 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:24,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:24,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,928 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-02 03:54:24,928 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:24,928 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-02 03:54:24,928 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-02 03:54:24,929 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,929 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,930 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:24,930 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:24,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:24,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:24,930 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:24,930 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:24,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,931 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:24,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,931 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,931 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-02 03:54:24,931 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:24,931 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-02 03:54:24,931 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:24,931 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-02 03:54:24,932 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:24,932 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:24,932 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:24,932 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. started... 2023-11-02 03:54:24,932 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 06d225f5eeba431dae3475159f87d35e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:25,344 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101aa3e88b372a04c16a950ac9ebbc410c5 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101aa3e88b372a04c16a950ac9ebbc410c5 2023-11-02 03:54:25,749 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/0a9b17d2cd3e4fa09557ceee9a7da61e 2023-11-02 03:54:25,754 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/0a9b17d2cd3e4fa09557ceee9a7da61e as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/0a9b17d2cd3e4fa09557ceee9a7da61e 2023-11-02 03:54:25,758 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/0a9b17d2cd3e4fa09557ceee9a7da61e, entries=1, sequenceid=29, filesize=5.0 K 2023-11-02 03:54:25,758 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 06d225f5eeba431dae3475159f87d35e in 826ms, sequenceid=29, compaction requested=true 2023-11-02 03:54:25,758 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:54:25,758 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:25,758 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:25,758 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:25,759 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:25,759 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-02 03:54:25,759 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-02 03:54:25,760 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:25,760 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:25,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:25,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:25,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:25,761 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,761 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:25,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:25,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:25,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:25,762 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-02 03:54:25,762 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7000ecdf[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:25,762 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-02 03:54:25,762 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,764 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,764 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-02 03:54:25,764 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-02 03:54:25,764 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:25,764 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:25,764 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:25,769 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-02 03:54:25,769 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:25,769 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:25,769 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-02 03:54:25,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:25,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:25,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:25,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:25,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:25,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:25,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:25,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:25,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:25,771 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-02 03:54:25,771 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-02 03:54:25,771 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:25,771 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:25,771 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,771 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:25,773 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,773 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:25,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:25,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,773 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:25,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:25,773 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:25,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:25,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:25,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:25,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,775 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:25,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:25,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:25,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:25,777 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:25,777 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:25,777 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,777 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:25,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:25,777 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-02 03:54:25,777 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:25,777 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-02 03:54:25,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:25,777 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:25,777 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,778 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:25,778 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,778 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:25,778 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:25,778 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,778 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:25,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:25,878 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:25,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:25,882 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:25,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:25,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,884 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-02 03:54:25,884 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:25,884 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-02 03:54:25,884 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-02 03:54:25,884 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,884 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,886 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:25,886 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:25,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:25,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:25,886 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:25,886 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:25,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,886 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:25,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,887 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,887 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-02 03:54:25,887 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:25,887 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-02 03:54:25,887 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:25,887 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-02 03:54:25,887 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:25,887 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:25,887 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:25,887 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. started... 2023-11-02 03:54:25,887 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 06d225f5eeba431dae3475159f87d35e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:26,299 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601039cda592a21934f189fe06a24bc0a3887 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601039cda592a21934f189fe06a24bc0a3887 2023-11-02 03:54:26,705 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/7d7dbcb3e92c4305aa7bac6aeeb0d5aa 2023-11-02 03:54:26,709 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/7d7dbcb3e92c4305aa7bac6aeeb0d5aa as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/7d7dbcb3e92c4305aa7bac6aeeb0d5aa 2023-11-02 03:54:26,713 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/7d7dbcb3e92c4305aa7bac6aeeb0d5aa, entries=1, sequenceid=33, filesize=5.0 K 2023-11-02 03:54:26,714 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 06d225f5eeba431dae3475159f87d35e in 827ms, sequenceid=33, compaction requested=true 2023-11-02 03:54:26,714 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:54:26,714 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:26,714 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:26,714 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:26,714 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:26,714 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-02 03:54:26,714 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-02 03:54:26,717 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:26,717 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:26,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:26,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:26,717 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,718 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:26,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:26,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:26,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:26,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:26,719 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-02 03:54:26,719 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@a9ca4b6[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:26,719 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-02 03:54:26,719 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,720 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,720 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-02 03:54:26,720 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-02 03:54:26,720 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:26,720 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:26,720 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:26,722 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:26,722 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-02 03:54:26,722 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:26,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:26,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:26,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:26,722 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-02 03:54:26,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:26,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:26,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:26,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:26,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:26,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:26,724 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-02 03:54:26,724 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-02 03:54:26,724 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:26,724 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:26,724 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,724 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:26,725 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,726 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,726 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:26,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:26,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:26,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:26,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:26,726 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:26,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:26,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:26,727 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:26,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:26,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:26,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:26,736 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:26,736 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,736 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:26,736 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:26,736 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:26,736 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-02 03:54:26,736 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:26,736 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:26,736 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,736 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:26,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:26,736 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,736 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:26,736 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-02 03:54:26,736 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,736 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:26,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:26,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:26,836 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:26,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:26,841 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:26,842 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:26,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,843 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-02 03:54:26,843 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:26,843 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-02 03:54:26,843 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-02 03:54:26,843 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,843 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,844 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:26,844 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:26,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:26,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:26,845 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:26,845 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:26,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,845 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:26,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,845 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,846 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-02 03:54:26,846 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:26,846 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-02 03:54:26,846 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:26,846 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-02 03:54:26,846 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:26,846 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:26,846 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:26,846 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. started... 2023-11-02 03:54:26,846 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 06d225f5eeba431dae3475159f87d35e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:27,258 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509079781318b3f5b4c35b4ce5726d54d4dd1 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201509079781318b3f5b4c35b4ce5726d54d4dd1 2023-11-02 03:54:27,664 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/3c55275b8f834eed9ecf423b6f206009 2023-11-02 03:54:27,669 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/3c55275b8f834eed9ecf423b6f206009 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/3c55275b8f834eed9ecf423b6f206009 2023-11-02 03:54:27,672 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/3c55275b8f834eed9ecf423b6f206009, entries=1, sequenceid=37, filesize=5.0 K 2023-11-02 03:54:27,673 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 06d225f5eeba431dae3475159f87d35e in 827ms, sequenceid=37, compaction requested=true 2023-11-02 03:54:27,673 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:54:27,673 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:27,673 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:27,674 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:27,674 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:27,674 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-02 03:54:27,674 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-02 03:54:27,675 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:27,675 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:27,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:27,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:27,676 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:27,676 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:27,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:27,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:27,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:27,677 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-02 03:54:27,677 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@34934aed[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:27,677 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-02 03:54:27,677 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,679 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,679 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-02 03:54:27,679 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-02 03:54:27,679 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:27,679 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:27,679 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:27,681 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:27,681 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-02 03:54:27,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:27,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:27,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:27,681 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:27,681 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-02 03:54:27,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:27,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:27,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:27,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:27,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:27,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:27,683 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-02 03:54:27,683 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-02 03:54:27,683 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:27,683 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:27,683 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,684 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:27,685 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,685 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:27,685 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:27,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:27,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:27,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:27,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:27,686 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:27,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:27,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:27,687 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:27,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:27,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:27,693 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:27,693 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,693 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:27,693 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,693 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-02 03:54:27,693 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:27,693 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:27,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:27,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:27,693 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:27,693 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:27,693 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:27,693 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:27,693 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,693 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,693 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:27,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:27,793 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:27,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:27,798 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:27,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:27,799 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,799 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-02 03:54:27,799 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:27,799 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-11-02 03:54:27,799 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-11-02 03:54:27,800 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,800 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,801 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:27,801 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:27,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:27,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:27,801 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:27,801 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:27,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,802 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:27,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,802 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,802 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-11-02 03:54:27,802 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:27,802 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-11-02 03:54:27,802 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:27,803 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-11-02 03:54:27,803 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:27,803 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:27,803 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:27,803 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. started... 2023-11-02 03:54:27,803 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 06d225f5eeba431dae3475159f87d35e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:28,215 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120022cd72b00794a9f962f85831b83b4be to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120022cd72b00794a9f962f85831b83b4be 2023-11-02 03:54:28,621 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/e56c467f8dab4b57a10de3b2251f67a2 2023-11-02 03:54:28,627 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/e56c467f8dab4b57a10de3b2251f67a2 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/e56c467f8dab4b57a10de3b2251f67a2 2023-11-02 03:54:28,631 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/e56c467f8dab4b57a10de3b2251f67a2, entries=1, sequenceid=41, filesize=5.0 K 2023-11-02 03:54:28,632 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 06d225f5eeba431dae3475159f87d35e in 829ms, sequenceid=41, compaction requested=true 2023-11-02 03:54:28,632 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:54:28,632 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:28,632 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:28,632 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:28,632 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:28,632 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-11-02 03:54:28,632 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-11-02 03:54:28,634 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:28,634 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:28,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:28,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:28,634 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,634 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:28,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:28,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:28,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:28,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:28,635 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-11-02 03:54:28,635 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6819dcc6[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:28,635 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-11-02 03:54:28,635 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,636 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,637 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-11-02 03:54:28,637 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-11-02 03:54:28,637 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:28,637 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:28,637 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:28,639 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:28,639 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-11-02 03:54:28,639 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:28,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:28,639 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-11-02 03:54:28,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:28,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:28,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:28,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:28,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:28,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:28,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:28,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:28,641 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-11-02 03:54:28,641 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-11-02 03:54:28,641 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:28,641 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:28,641 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,641 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:28,643 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,643 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,643 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:28,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:28,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:28,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:28,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:28,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:28,643 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:28,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:28,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,644 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:28,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:28,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:28,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:28,647 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:28,647 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,647 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,647 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:28,647 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,647 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,647 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,647 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:28,647 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-11-02 03:54:28,647 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:28,647 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-11-02 03:54:28,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:28,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:28,647 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:28,647 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:28,647 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:28,647 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:28,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:28,647 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:28,748 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:28,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:28,750 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41275] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:28,751 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] procedure2.ProcedureExecutor(1028): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-02 03:54:28,752 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-02 03:54:28,752 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:54:28,752 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698897868752 2023-11-02 03:54:28,753 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-02 03:54:28,754 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-11-02 03:54:28,754 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-02 03:54:28,756 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] zookeeper.ReadOnlyZKClient(139): Connect 0x57bdaa3f to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:54:28,760 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@45c2680e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:54:28,760 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:29,197 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103b45dc5fa3157455bb9f79a8f7f694c0c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103b45dc5fa3157455bb9f79a8f7f694c0c 2023-11-02 03:54:29,197 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-02 03:54:29,210 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-02 03:54:29,211 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48644, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-02 03:54:29,217 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-02 03:54:29,218 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52856, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-02 03:54:29,225 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/f80d79e61b4d45459bd2559469b6afa8 first=Optional[r3] last=Optional[r6] 2023-11-02 03:54:29,226 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/f80d79e61b4d45459bd2559469b6afa8}] 2023-11-02 03:54:29,230 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/f80d79e61b4d45459bd2559469b6afa8 for inclusion in 06d225f5eeba431dae3475159f87d35e/family1 2023-11-02 03:54:29,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-11-02 03:54:29,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last= 2023-11-02 03:54:29,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:54:29,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/f80d79e61b4d45459bd2559469b6afa8 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__hgaf53lf3q5j6jff78mas8sicvog4u394m59gotk2t7n1iskugjq1lp06o10rhh0/family1/f80d79e61b4d45459bd2559469b6afa8 2023-11-02 03:54:29,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__hgaf53lf3q5j6jff78mas8sicvog4u394m59gotk2t7n1iskugjq1lp06o10rhh0/family1/f80d79e61b4d45459bd2559469b6afa8 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/5c143586eb334f33b92cc1db7c8be92a_SeqId_44_ 2023-11-02 03:54:29,235 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__hgaf53lf3q5j6jff78mas8sicvog4u394m59gotk2t7n1iskugjq1lp06o10rhh0/family1/f80d79e61b4d45459bd2559469b6afa8 into 06d225f5eeba431dae3475159f87d35e/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/5c143586eb334f33b92cc1db7c8be92a_SeqId_44_ - updating store file list. 2023-11-02 03:54:29,238 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/5c143586eb334f33b92cc1db7c8be92a_SeqId_44_ into 06d225f5eeba431dae3475159f87d35e/family1 2023-11-02 03:54:29,238 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__hgaf53lf3q5j6jff78mas8sicvog4u394m59gotk2t7n1iskugjq1lp06o10rhh0/family1/f80d79e61b4d45459bd2559469b6afa8 into 06d225f5eeba431dae3475159f87d35e/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/5c143586eb334f33b92cc1db7c8be92a_SeqId_44_) 2023-11-02 03:54:29,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__hgaf53lf3q5j6jff78mas8sicvog4u394m59gotk2t7n1iskugjq1lp06o10rhh0/family1/f80d79e61b4d45459bd2559469b6afa8 2023-11-02 03:54:29,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-02 03:54:29,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-02 03:54:29,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(1912): 06d225f5eeba431dae3475159f87d35e/family1 is initiating minor compaction 2023-11-02 03:54:29,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@80204c8; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:54:29,242 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 06d225f5eeba431dae3475159f87d35e/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:29,242 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/a4bdf8329d5e4137986b41aa073a3806, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/2420df720bf345c6a6779696b1c9aab5, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/ed26775f24b44d49acf7bd87b1ded286, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/eb3bc0ade0f64e24acccb673678e302c, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/d9f111deb459406aaa770aecf31c5f49, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/0a9b17d2cd3e4fa09557ceee9a7da61e, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/7d7dbcb3e92c4305aa7bac6aeeb0d5aa, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/3c55275b8f834eed9ecf423b6f206009, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/e56c467f8dab4b57a10de3b2251f67a2, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/5c143586eb334f33b92cc1db7c8be92a_SeqId_44_] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp, totalSize=50.4 K 2023-11-02 03:54:29,242 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting a4bdf8329d5e4137986b41aa073a3806, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-02 03:54:29,242 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 2420df720bf345c6a6779696b1c9aab5, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-02 03:54:29,242 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting ed26775f24b44d49acf7bd87b1ded286, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-02 03:54:29,242 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting eb3bc0ade0f64e24acccb673678e302c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-02 03:54:29,242 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting d9f111deb459406aaa770aecf31c5f49, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-02 03:54:29,242 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 0a9b17d2cd3e4fa09557ceee9a7da61e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-02 03:54:29,242 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 7d7dbcb3e92c4305aa7bac6aeeb0d5aa, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-02 03:54:29,242 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 3c55275b8f834eed9ecf423b6f206009, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-02 03:54:29,242 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting e56c467f8dab4b57a10de3b2251f67a2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-02 03:54:29,243 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 5c143586eb334f33b92cc1db7c8be92a_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-02 03:54:29,243 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-02 03:54:29,244 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:54:29,246 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512287d66442bb2e243fcbcd97943b7017772 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512287d66442bb2e243fcbcd97943b7017772 2023-11-02 03:54:29,247 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231810fcbc36d9f4fc5b2adc455b46ebe2d to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231810fcbc36d9f4fc5b2adc455b46ebe2d 2023-11-02 03:54:29,248 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101aa3e88b372a04c16a950ac9ebbc410c5 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160101aa3e88b372a04c16a950ac9ebbc410c5 2023-11-02 03:54:29,249 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601039cda592a21934f189fe06a24bc0a3887 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601039cda592a21934f189fe06a24bc0a3887 2023-11-02 03:54:29,249 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-11-02 03:54:29,249 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-11-02 03:54:29,250 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:29,250 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:29,250 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:29,250 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:29,250 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:29,250 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:29,250 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:29,250 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:29,265 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] throttle.PressureAwareThroughputController(145): 06d225f5eeba431dae3475159f87d35e#family1#compaction#60 average throughput is 1.59 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-02 03:54:30,080 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/fe9aa616a4aa41f4a39f8701f2528711 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/fe9aa616a4aa41f4a39f8701f2528711 2023-11-02 03:54:30,086 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 06d225f5eeba431dae3475159f87d35e/family1 of 06d225f5eeba431dae3475159f87d35e into fe9aa616a4aa41f4a39f8701f2528711(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-02 03:54:30,086 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:54:30,086 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1698897269242; duration=0sec 2023-11-02 03:54:30,087 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:54:30,089 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205d68e003595b7486fbbb4cd17ce163538 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205d68e003595b7486fbbb4cd17ce163538 2023-11-02 03:54:30,089 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-02 03:54:30,113 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9dff4170870042809518d58da05181cc first=Optional[r00] last=Optional[r2] 2023-11-02 03:54:30,114 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9dff4170870042809518d58da05181cc}] 2023-11-02 03:54:30,115 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9dff4170870042809518d58da05181cc for inclusion in 06d225f5eeba431dae3475159f87d35e/family1 2023-11-02 03:54:30,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-11-02 03:54:30,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last= 2023-11-02 03:54:30,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:54:30,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/9dff4170870042809518d58da05181cc to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__2ta40c3imtfjruudl1ohr6mls7lu7d4t8t5gn7ibgmijh14s28cckkdr3d123rfg/family1/9dff4170870042809518d58da05181cc 2023-11-02 03:54:30,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__2ta40c3imtfjruudl1ohr6mls7lu7d4t8t5gn7ibgmijh14s28cckkdr3d123rfg/family1/9dff4170870042809518d58da05181cc as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/8014d2401b5f4523bf7d3a0f10857082_SeqId_47_ 2023-11-02 03:54:30,121 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__2ta40c3imtfjruudl1ohr6mls7lu7d4t8t5gn7ibgmijh14s28cckkdr3d123rfg/family1/9dff4170870042809518d58da05181cc into 06d225f5eeba431dae3475159f87d35e/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/8014d2401b5f4523bf7d3a0f10857082_SeqId_47_ - updating store file list. 2023-11-02 03:54:30,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/8014d2401b5f4523bf7d3a0f10857082_SeqId_47_ into 06d225f5eeba431dae3475159f87d35e/family1 2023-11-02 03:54:30,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__2ta40c3imtfjruudl1ohr6mls7lu7d4t8t5gn7ibgmijh14s28cckkdr3d123rfg/family1/9dff4170870042809518d58da05181cc into 06d225f5eeba431dae3475159f87d35e/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/8014d2401b5f4523bf7d3a0f10857082_SeqId_47_) 2023-11-02 03:54:30,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__2ta40c3imtfjruudl1ohr6mls7lu7d4t8t5gn7ibgmijh14s28cckkdr3d123rfg/family1/9dff4170870042809518d58da05181cc 2023-11-02 03:54:30,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-02 03:54:30,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-02 03:54:30,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(1912): 06d225f5eeba431dae3475159f87d35e/family1 is initiating minor compaction (all files) 2023-11-02 03:54:30,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@780995c6; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:54:30,126 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 06d225f5eeba431dae3475159f87d35e/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:30,126 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/05b186f8a71b4f2d9e9e71739bcd324f, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/fe9aa616a4aa41f4a39f8701f2528711, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/8014d2401b5f4523bf7d3a0f10857082_SeqId_47_] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp, totalSize=390.1 K 2023-11-02 03:54:30,126 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 05b186f8a71b4f2d9e9e71739bcd324f, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878824741 2023-11-02 03:54:30,126 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting fe9aa616a4aa41f4a39f8701f2528711, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621224741 2023-11-02 03:54:30,127 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 8014d2401b5f4523bf7d3a0f10857082_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878824741 2023-11-02 03:54:30,127 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-02 03:54:30,128 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:54:30,129 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130158e1b895ea04067bd813d734a099d12 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130158e1b895ea04067bd813d734a099d12 2023-11-02 03:54:30,130 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120187f35de2cdcc4313bedb673c83bc6db1 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120187f35de2cdcc4313bedb673c83bc6db1 2023-11-02 03:54:30,131 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205c0914d2f42b1422c89a8d09bdbf8d561 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205c0914d2f42b1422c89a8d09bdbf8d561 2023-11-02 03:54:30,131 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-11-02 03:54:30,131 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-11-02 03:54:30,132 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:30,132 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-02 03:54:30,132 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:54:30,133 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-02 03:54:30,134 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.3830 sec 2023-11-02 03:54:30,146 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] throttle.PressureAwareThroughputController(145): 06d225f5eeba431dae3475159f87d35e#family1#compaction#61 average throughput is 16.42 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-02 03:54:30,168 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/a4bdf8329d5e4137986b41aa073a3806, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/2420df720bf345c6a6779696b1c9aab5, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/ed26775f24b44d49acf7bd87b1ded286, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/eb3bc0ade0f64e24acccb673678e302c, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/d9f111deb459406aaa770aecf31c5f49, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/0a9b17d2cd3e4fa09557ceee9a7da61e, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/7d7dbcb3e92c4305aa7bac6aeeb0d5aa, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/3c55275b8f834eed9ecf423b6f206009, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/e56c467f8dab4b57a10de3b2251f67a2, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/5c143586eb334f33b92cc1db7c8be92a_SeqId_44_] to archive 2023-11-02 03:54:30,169 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:54:30,172 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/a4bdf8329d5e4137986b41aa073a3806 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/a4bdf8329d5e4137986b41aa073a3806 2023-11-02 03:54:30,174 DEBUG [hfile_cleaner-dir-scan-pool-0] cleaner.CleanerChore(483): Couldn't delete 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' yet because it isn't empty w/exception. org.apache.hadoop.fs.PathIsNotEmptyDirectoryException: ``/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) ': Directory is not empty: `/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1595) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:881) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:878) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:878) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$4(CleanerChore.java:453) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:478) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:453) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:61) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:439) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$2(CleanerChore.java:432) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): `/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.delete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:572) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1593) ... 29 more 2023-11-02 03:54:30,174 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/2420df720bf345c6a6779696b1c9aab5 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/2420df720bf345c6a6779696b1c9aab5 2023-11-02 03:54:30,176 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/ed26775f24b44d49acf7bd87b1ded286 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/ed26775f24b44d49acf7bd87b1ded286 2023-11-02 03:54:30,177 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/eb3bc0ade0f64e24acccb673678e302c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/eb3bc0ade0f64e24acccb673678e302c 2023-11-02 03:54:30,179 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/d9f111deb459406aaa770aecf31c5f49 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/d9f111deb459406aaa770aecf31c5f49 2023-11-02 03:54:30,180 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/0a9b17d2cd3e4fa09557ceee9a7da61e to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/0a9b17d2cd3e4fa09557ceee9a7da61e 2023-11-02 03:54:30,182 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/7d7dbcb3e92c4305aa7bac6aeeb0d5aa to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/7d7dbcb3e92c4305aa7bac6aeeb0d5aa 2023-11-02 03:54:30,184 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/3c55275b8f834eed9ecf423b6f206009 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/3c55275b8f834eed9ecf423b6f206009 2023-11-02 03:54:30,185 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/e56c467f8dab4b57a10de3b2251f67a2 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/e56c467f8dab4b57a10de3b2251f67a2 2023-11-02 03:54:30,186 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/5c143586eb334f33b92cc1db7c8be92a_SeqId_44_ to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/5c143586eb334f33b92cc1db7c8be92a_SeqId_44_ 2023-11-02 03:54:30,220 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-11-02 03:54:30,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:30,629 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-11-02 03:54:30,630 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-11-02 03:54:30,630 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-11-02 03:54:30,632 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-11-02 03:54:30,635 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=06d225f5eeba431dae3475159f87d35e, REOPEN/MOVE}] 2023-11-02 03:54:30,636 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=06d225f5eeba431dae3475159f87d35e, REOPEN/MOVE 2023-11-02 03:54:30,636 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=06d225f5eeba431dae3475159f87d35e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:30,636 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698897270636"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897270636"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897270636"}]},"ts":"1698897270636"} 2023-11-02 03:54:30,638 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 06d225f5eeba431dae3475159f87d35e, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:54:30,789 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:30,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:30,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 06d225f5eeba431dae3475159f87d35e, disabling compactions & flushes 2023-11-02 03:54:30,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:31,364 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/.tmp/family1/d26bbb3cfa2a4f67a4fa367a36e3973e as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/d26bbb3cfa2a4f67a4fa367a36e3973e 2023-11-02 03:54:31,368 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 06d225f5eeba431dae3475159f87d35e/family1 of 06d225f5eeba431dae3475159f87d35e into d26bbb3cfa2a4f67a4fa367a36e3973e(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-02 03:54:31,368 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:54:31,368 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:31,368 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1698897270125; duration=1sec 2023-11-02 03:54:31,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:31,368 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:54:31,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. after waiting 0 ms 2023-11-02 03:54:31,368 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:31,369 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/05b186f8a71b4f2d9e9e71739bcd324f, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/fe9aa616a4aa41f4a39f8701f2528711, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/8014d2401b5f4523bf7d3a0f10857082_SeqId_47_] to archive 2023-11-02 03:54:31,369 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:54:31,371 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/05b186f8a71b4f2d9e9e71739bcd324f to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/05b186f8a71b4f2d9e9e71739bcd324f 2023-11-02 03:54:31,372 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/fe9aa616a4aa41f4a39f8701f2528711 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/fe9aa616a4aa41f4a39f8701f2528711 2023-11-02 03:54:31,373 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/8014d2401b5f4523bf7d3a0f10857082_SeqId_47_ to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/8014d2401b5f4523bf7d3a0f10857082_SeqId_47_ 2023-11-02 03:54:31,377 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-11-02 03:54:31,378 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:31,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:54:31,378 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3510): Not adding moved region record: 06d225f5eeba431dae3475159f87d35e to self. 2023-11-02 03:54:31,380 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:31,380 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=06d225f5eeba431dae3475159f87d35e, regionState=CLOSED 2023-11-02 03:54:31,380 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698897271380"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897271380"}]},"ts":"1698897271380"} 2023-11-02 03:54:31,382 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=46, resume processing ppid=45 2023-11-02 03:54:31,383 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 06d225f5eeba431dae3475159f87d35e, server=jenkins-hbase4.apache.org,36627,1698897201574 in 743 msec 2023-11-02 03:54:31,383 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=06d225f5eeba431dae3475159f87d35e, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=true 2023-11-02 03:54:31,533 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=06d225f5eeba431dae3475159f87d35e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:31,534 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698897271533"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897271533"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897271533"}]},"ts":"1698897271533"} 2023-11-02 03:54:31,535 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure 06d225f5eeba431dae3475159f87d35e, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:54:31,686 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:31,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:31,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 06d225f5eeba431dae3475159f87d35e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.', STARTKEY => '', ENDKEY => ''} 2023-11-02 03:54:31,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:31,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:31,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:31,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:31,690 INFO [StoreOpener-06d225f5eeba431dae3475159f87d35e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:31,691 DEBUG [StoreOpener-06d225f5eeba431dae3475159f87d35e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1 2023-11-02 03:54:31,691 DEBUG [StoreOpener-06d225f5eeba431dae3475159f87d35e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1 2023-11-02 03:54:31,691 INFO [StoreOpener-06d225f5eeba431dae3475159f87d35e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 06d225f5eeba431dae3475159f87d35e columnFamilyName family1 2023-11-02 03:54:31,696 DEBUG [StoreOpener-06d225f5eeba431dae3475159f87d35e-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/d26bbb3cfa2a4f67a4fa367a36e3973e 2023-11-02 03:54:31,696 INFO [StoreOpener-06d225f5eeba431dae3475159f87d35e-1] regionserver.HStore(310): Store=06d225f5eeba431dae3475159f87d35e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:54:31,697 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:31,698 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:31,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:31,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 06d225f5eeba431dae3475159f87d35e; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73104411, jitterRate=0.08934061229228973}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-02 03:54:31,701 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:54:31,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e., pid=47, masterSystemTime=1698897271686 2023-11-02 03:54:31,703 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:31,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:31,703 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=06d225f5eeba431dae3475159f87d35e, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:31,703 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698897271703"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897271703"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897271703"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897271703"}]},"ts":"1698897271703"} 2023-11-02 03:54:31,706 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=47, resume processing ppid=45 2023-11-02 03:54:31,706 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure 06d225f5eeba431dae3475159f87d35e, server=jenkins-hbase4.apache.org,36627,1698897201574 in 170 msec 2023-11-02 03:54:31,707 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=45, resume processing ppid=44 2023-11-02 03:54:31,708 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=06d225f5eeba431dae3475159f87d35e, REOPEN/MOVE in 1.0710 sec 2023-11-02 03:54:31,709 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=44, resume processing ppid=43 2023-11-02 03:54:31,709 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.0760 sec 2023-11-02 03:54:31,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.4900 sec 2023-11-02 03:54:31,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=43 2023-11-02 03:54:31,712 INFO [Listener at localhost/43647] compactions.TestMobCompactor(278): alter status finished 2023-11-02 03:54:31,713 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41275] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:31,714 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] procedure2.ProcedureExecutor(1028): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-02 03:54:31,715 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-02 03:54:31,715 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:54:31,715 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698897871715 2023-11-02 03:54:31,716 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-02 03:54:31,717 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-11-02 03:54:31,717 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-02 03:54:31,718 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] zookeeper.ReadOnlyZKClient(139): Connect 0x6139e3b2 to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:54:31,722 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@37ed6583, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:54:31,723 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:31,723 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:31,723 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:31,723 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:31,723 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:32,549 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128a59fbd93feeb4683b58ec315b0eb9ac1 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128a59fbd93feeb4683b58ec315b0eb9ac1 2023-11-02 03:54:32,550 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-02 03:54:32,569 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-02 03:54:32,570 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48646, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-02 03:54:32,576 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-02 03:54:32,578 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52862, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-02 03:54:32,587 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/607d860f2cbc4e41a19e4b1801612e1b first=Optional[r05] last=Optional[r8] 2023-11-02 03:54:32,587 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=51 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/607d860f2cbc4e41a19e4b1801612e1b}] 2023-11-02 03:54:32,591 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/607d860f2cbc4e41a19e4b1801612e1b for inclusion in 06d225f5eeba431dae3475159f87d35e/family1 2023-11-02 03:54:32,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-11-02 03:54:32,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last= 2023-11-02 03:54:32,594 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:54:32,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/607d860f2cbc4e41a19e4b1801612e1b to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__uojhsln0qiirnh9jvlgb2q3a96c357837uvkna68bj3kcbsrm0sbt3si7qmtl0em/family1/607d860f2cbc4e41a19e4b1801612e1b 2023-11-02 03:54:32,598 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__uojhsln0qiirnh9jvlgb2q3a96c357837uvkna68bj3kcbsrm0sbt3si7qmtl0em/family1/607d860f2cbc4e41a19e4b1801612e1b as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/f7ab521b45674d518a564d0b356f51b3_SeqId_53_ 2023-11-02 03:54:32,598 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__uojhsln0qiirnh9jvlgb2q3a96c357837uvkna68bj3kcbsrm0sbt3si7qmtl0em/family1/607d860f2cbc4e41a19e4b1801612e1b into 06d225f5eeba431dae3475159f87d35e/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/f7ab521b45674d518a564d0b356f51b3_SeqId_53_ - updating store file list. 2023-11-02 03:54:32,602 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/f7ab521b45674d518a564d0b356f51b3_SeqId_53_ into 06d225f5eeba431dae3475159f87d35e/family1 2023-11-02 03:54:32,602 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__uojhsln0qiirnh9jvlgb2q3a96c357837uvkna68bj3kcbsrm0sbt3si7qmtl0em/family1/607d860f2cbc4e41a19e4b1801612e1b into 06d225f5eeba431dae3475159f87d35e/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/f7ab521b45674d518a564d0b356f51b3_SeqId_53_) 2023-11-02 03:54:32,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__uojhsln0qiirnh9jvlgb2q3a96c357837uvkna68bj3kcbsrm0sbt3si7qmtl0em/family1/607d860f2cbc4e41a19e4b1801612e1b 2023-11-02 03:54:32,604 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-02 03:54:32,605 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:54:32,607 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120022cd72b00794a9f962f85831b83b4be to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120022cd72b00794a9f962f85831b83b4be 2023-11-02 03:54:32,608 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511286f905254e253434d9e2ddbb831035e56 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511286f905254e253434d9e2ddbb831035e56 2023-11-02 03:54:32,608 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-11-02 03:54:32,608 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-11-02 03:54:32,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:32,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:32,608 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:32,608 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-02 03:54:32,608 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:54:32,609 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-02 03:54:32,611 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 895 msec 2023-11-02 03:54:32,667 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-02 03:54:32,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:32,677 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-11-02 03:54:32,678 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-11-02 03:54:32,678 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-11-02 03:54:32,680 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-11-02 03:54:32,683 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=06d225f5eeba431dae3475159f87d35e, REOPEN/MOVE}] 2023-11-02 03:54:32,684 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=06d225f5eeba431dae3475159f87d35e, REOPEN/MOVE 2023-11-02 03:54:32,684 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=06d225f5eeba431dae3475159f87d35e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:32,685 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698897272684"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897272684"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897272684"}]},"ts":"1698897272684"} 2023-11-02 03:54:32,686 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure 06d225f5eeba431dae3475159f87d35e, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:54:32,837 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:32,838 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:32,838 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 06d225f5eeba431dae3475159f87d35e, disabling compactions & flushes 2023-11-02 03:54:32,838 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:32,838 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:32,838 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. after waiting 0 ms 2023-11-02 03:54:32,838 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:32,843 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-11-02 03:54:32,844 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:32,844 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:54:32,844 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3510): Not adding moved region record: 06d225f5eeba431dae3475159f87d35e to self. 2023-11-02 03:54:32,846 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:32,846 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=06d225f5eeba431dae3475159f87d35e, regionState=CLOSED 2023-11-02 03:54:32,846 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698897272846"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897272846"}]},"ts":"1698897272846"} 2023-11-02 03:54:32,848 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=52, resume processing ppid=51 2023-11-02 03:54:32,848 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure 06d225f5eeba431dae3475159f87d35e, server=jenkins-hbase4.apache.org,36627,1698897201574 in 161 msec 2023-11-02 03:54:32,849 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=06d225f5eeba431dae3475159f87d35e, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=true 2023-11-02 03:54:32,999 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=06d225f5eeba431dae3475159f87d35e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:32,999 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698897272999"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897272999"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897272999"}]},"ts":"1698897272999"} 2023-11-02 03:54:33,001 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure 06d225f5eeba431dae3475159f87d35e, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:54:33,152 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:33,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:33,155 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 06d225f5eeba431dae3475159f87d35e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.', STARTKEY => '', ENDKEY => ''} 2023-11-02 03:54:33,155 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:33,155 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:33,156 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:33,156 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:33,157 INFO [StoreOpener-06d225f5eeba431dae3475159f87d35e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:33,158 DEBUG [StoreOpener-06d225f5eeba431dae3475159f87d35e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1 2023-11-02 03:54:33,158 DEBUG [StoreOpener-06d225f5eeba431dae3475159f87d35e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1 2023-11-02 03:54:33,158 INFO [StoreOpener-06d225f5eeba431dae3475159f87d35e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 06d225f5eeba431dae3475159f87d35e columnFamilyName family1 2023-11-02 03:54:33,163 DEBUG [StoreOpener-06d225f5eeba431dae3475159f87d35e-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/d26bbb3cfa2a4f67a4fa367a36e3973e 2023-11-02 03:54:33,166 DEBUG [StoreOpener-06d225f5eeba431dae3475159f87d35e-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/family1/f7ab521b45674d518a564d0b356f51b3_SeqId_53_ 2023-11-02 03:54:33,166 INFO [StoreOpener-06d225f5eeba431dae3475159f87d35e-1] regionserver.HStore(310): Store=06d225f5eeba431dae3475159f87d35e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:54:33,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:33,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:33,171 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 06d225f5eeba431dae3475159f87d35e 2023-11-02 03:54:33,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 06d225f5eeba431dae3475159f87d35e; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71041526, jitterRate=0.05860123038291931}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-02 03:54:33,172 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:54:33,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e., pid=53, masterSystemTime=1698897273152 2023-11-02 03:54:33,174 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:33,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:54:33,174 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=06d225f5eeba431dae3475159f87d35e, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:33,174 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1698897273174"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897273174"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897273174"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897273174"}]},"ts":"1698897273174"} 2023-11-02 03:54:33,177 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=53, resume processing ppid=51 2023-11-02 03:54:33,177 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure 06d225f5eeba431dae3475159f87d35e, server=jenkins-hbase4.apache.org,36627,1698897201574 in 174 msec 2023-11-02 03:54:33,178 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=51, resume processing ppid=50 2023-11-02 03:54:33,178 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=06d225f5eeba431dae3475159f87d35e, REOPEN/MOVE in 494 msec 2023-11-02 03:54:33,180 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=50, resume processing ppid=49 2023-11-02 03:54:33,180 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 498 msec 2023-11-02 03:54:33,181 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 513 msec 2023-11-02 03:54:33,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-11-02 03:54:33,183 INFO [Listener at localhost/43647] compactions.TestMobCompactor(278): alter status finished 2023-11-02 03:54:33,184 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41275] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-11-02 03:54:33,185 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] procedure2.ProcedureExecutor(1028): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-02 03:54:33,185 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-02 03:54:33,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:54:33,186 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698897873186 2023-11-02 03:54:33,187 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-02 03:54:33,188 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-11-02 03:54:33,188 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-02 03:54:33,189 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] zookeeper.ReadOnlyZKClient(139): Connect 0x30457c61 to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:54:33,193 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6c58d697, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:54:33,193 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:33,193 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:33,193 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:33,193 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:33,193 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:33,193 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:33,193 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:33,193 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-11-02 03:54:33,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-11-02 03:54:33,193 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:54:33,194 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-11-02 03:54:33,195 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 10 msec 2023-11-02 03:54:33,241 INFO [Listener at localhost/43647] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=252 (was 264), OpenFileDescriptor=660 (was 656) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=175 (was 225), ProcessCount=170 (was 170), AvailableMemoryMB=11214 (was 11222) 2023-11-02 03:54:33,252 INFO [Listener at localhost/43647] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=252, OpenFileDescriptor=660, MaxFileDescriptor=60000, SystemLoadAverage=175, ProcessCount=170, AvailableMemoryMB=11213 2023-11-02 03:54:33,254 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-11-02 03:54:33,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:33,255 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-11-02 03:54:33,256 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-11-02 03:54:33,256 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-02 03:54:33,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-02 03:54:33,258 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:33,258 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e empty. 2023-11-02 03:54:33,259 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:33,259 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-11-02 03:54:33,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-02 03:54:33,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-02 03:54:33,667 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-11-02 03:54:33,669 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7675): creating {ENCODED => c9354336ff3eea83ac372fc04a8d438e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp 2023-11-02 03:54:33,859 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-02 03:54:34,074 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:34,075 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1604): Closing c9354336ff3eea83ac372fc04a8d438e, disabling compactions & flushes 2023-11-02 03:54:34,075 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:34,075 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:34,075 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. after waiting 0 ms 2023-11-02 03:54:34,075 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:34,075 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:34,075 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1558): Region close journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:54:34,077 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-11-02 03:54:34,078 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698897274078"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897274078"}]},"ts":"1698897274078"} 2023-11-02 03:54:34,079 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-02 03:54:34,080 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-02 03:54:34,080 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897274080"}]},"ts":"1698897274080"} 2023-11-02 03:54:34,080 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-11-02 03:54:34,084 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=c9354336ff3eea83ac372fc04a8d438e, ASSIGN}] 2023-11-02 03:54:34,086 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=c9354336ff3eea83ac372fc04a8d438e, ASSIGN 2023-11-02 03:54:34,087 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=c9354336ff3eea83ac372fc04a8d438e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=false 2023-11-02 03:54:34,237 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=c9354336ff3eea83ac372fc04a8d438e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:34,238 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698897274237"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897274237"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897274237"}]},"ts":"1698897274237"} 2023-11-02 03:54:34,239 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure c9354336ff3eea83ac372fc04a8d438e, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:54:34,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-02 03:54:34,390 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:34,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:34,394 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c9354336ff3eea83ac372fc04a8d438e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.', STARTKEY => '', ENDKEY => ''} 2023-11-02 03:54:34,394 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:34,394 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:34,394 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:34,394 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:34,396 INFO [StoreOpener-c9354336ff3eea83ac372fc04a8d438e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:34,398 DEBUG [StoreOpener-c9354336ff3eea83ac372fc04a8d438e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1 2023-11-02 03:54:34,398 DEBUG [StoreOpener-c9354336ff3eea83ac372fc04a8d438e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1 2023-11-02 03:54:34,398 INFO [StoreOpener-c9354336ff3eea83ac372fc04a8d438e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c9354336ff3eea83ac372fc04a8d438e columnFamilyName family1 2023-11-02 03:54:34,400 INFO [StoreOpener-c9354336ff3eea83ac372fc04a8d438e-1] regionserver.HStore(310): Store=c9354336ff3eea83ac372fc04a8d438e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:54:34,400 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:34,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:34,404 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:34,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:54:34,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c9354336ff3eea83ac372fc04a8d438e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74819244, jitterRate=0.11489361524581909}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-02 03:54:34,406 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:54:34,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e., pid=57, masterSystemTime=1698897274390 2023-11-02 03:54:34,408 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:34,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:34,408 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=c9354336ff3eea83ac372fc04a8d438e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:34,408 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698897274408"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897274408"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897274408"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897274408"}]},"ts":"1698897274408"} 2023-11-02 03:54:34,410 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=57, resume processing ppid=56 2023-11-02 03:54:34,411 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure c9354336ff3eea83ac372fc04a8d438e, server=jenkins-hbase4.apache.org,36627,1698897201574 in 170 msec 2023-11-02 03:54:34,412 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=56, resume processing ppid=55 2023-11-02 03:54:34,412 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=c9354336ff3eea83ac372fc04a8d438e, ASSIGN in 327 msec 2023-11-02 03:54:34,412 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-02 03:54:34,413 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897274413"}]},"ts":"1698897274413"} 2023-11-02 03:54:34,414 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-11-02 03:54:34,416 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-11-02 03:54:34,417 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1620 sec 2023-11-02 03:54:35,279 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:35,361 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=55 2023-11-02 03:54:35,361 INFO [Listener at localhost/43647] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-11-02 03:54:35,382 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:35,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:35,384 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:35,384 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:35,384 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-02 03:54:35,384 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-02 03:54:35,384 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:35,384 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:35,386 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:35,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:35,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:35,386 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:35,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:35,386 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:35,386 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:35,387 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:35,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:35,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:35,387 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:35,387 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:35,387 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-02 03:54:35,388 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:35,388 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-02 03:54:35,388 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:35,388 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:35,388 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:35,388 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. started... 2023-11-02 03:54:35,389 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c9354336ff3eea83ac372fc04a8d438e 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-02 03:54:35,807 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130eaa689dcaed04cf5b605c8b4dec562f0 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130eaa689dcaed04cf5b605c8b4dec562f0 2023-11-02 03:54:36,211 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/7a36b2104aff481393b84eb66326621c 2023-11-02 03:54:36,216 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/7a36b2104aff481393b84eb66326621c as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/7a36b2104aff481393b84eb66326621c 2023-11-02 03:54:36,220 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/7a36b2104aff481393b84eb66326621c, entries=1000, sequenceid=5, filesize=190.5 K 2023-11-02 03:54:36,220 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for c9354336ff3eea83ac372fc04a8d438e in 831ms, sequenceid=5, compaction requested=false 2023-11-02 03:54:36,221 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:54:36,221 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:36,221 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:36,221 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:36,221 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,221 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-02 03:54:36,221 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-02 03:54:36,223 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,223 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:36,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:36,223 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:36,223 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:36,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:36,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:36,225 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-02 03:54:36,225 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1f88b30e[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:36,225 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-02 03:54:36,225 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,226 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,226 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-02 03:54:36,226 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-02 03:54:36,226 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:36,226 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,226 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:36,228 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-02 03:54:36,228 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,228 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:36,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:36,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:36,228 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-02 03:54:36,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:36,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:36,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:36,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:36,230 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-02 03:54:36,230 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-02 03:54:36,231 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:36,231 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:36,231 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,231 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:36,243 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,243 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,243 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:36,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:36,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:36,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:36,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:36,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:36,244 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:36,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,245 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:36,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,248 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,248 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:36,248 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,248 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:36,248 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:36,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:36,248 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:36,248 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:36,248 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:36,248 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,248 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,248 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,248 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,248 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:36,248 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:36,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:36,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:36,348 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:36,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:36,353 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:36,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:36,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,354 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:36,354 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:36,354 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-02 03:54:36,354 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-02 03:54:36,355 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,355 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,357 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:36,357 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:36,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:36,357 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,357 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:36,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,358 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:36,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,358 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,358 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-02 03:54:36,358 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,358 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-02 03:54:36,358 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:36,358 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-02 03:54:36,359 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:36,359 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:36,359 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:36,359 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. started... 2023-11-02 03:54:36,359 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c9354336ff3eea83ac372fc04a8d438e 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-02 03:54:36,370 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128f09777a069b342878edffeb7c46f6e94 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128f09777a069b342878edffeb7c46f6e94 2023-11-02 03:54:36,776 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/43c2ee158cc2419a964e58f0b2ff5324 2023-11-02 03:54:36,780 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/43c2ee158cc2419a964e58f0b2ff5324 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/43c2ee158cc2419a964e58f0b2ff5324 2023-11-02 03:54:36,784 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/43c2ee158cc2419a964e58f0b2ff5324, entries=1, sequenceid=9, filesize=5.0 K 2023-11-02 03:54:36,785 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for c9354336ff3eea83ac372fc04a8d438e in 426ms, sequenceid=9, compaction requested=false 2023-11-02 03:54:36,785 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:54:36,785 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:36,785 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:36,785 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:36,785 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,785 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-02 03:54:36,785 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-02 03:54:36,787 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,787 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:36,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:36,787 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,787 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:36,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:36,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:36,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:36,789 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-02 03:54:36,789 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@70de1d5d[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:36,789 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-02 03:54:36,789 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,790 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,790 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-02 03:54:36,790 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-02 03:54:36,790 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:36,790 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,790 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:36,791 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-02 03:54:36,791 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:36,791 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,792 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-02 03:54:36,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:36,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:36,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:36,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:36,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:36,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:36,794 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-02 03:54:36,794 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-02 03:54:36,794 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:36,794 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:36,794 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,794 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:36,796 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,796 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:36,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:36,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,797 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:36,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:36,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:36,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:36,797 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:36,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,798 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:36,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,801 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,801 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:36,801 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:36,801 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:36,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:36,801 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:36,801 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:36,801 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:36,801 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,801 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,801 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,801 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:36,801 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:36,801 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:36,801 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:36,901 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:36,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:36,906 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:36,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:36,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,907 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:36,907 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:36,907 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-02 03:54:36,907 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-02 03:54:36,908 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,908 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,909 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,909 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:36,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:36,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:36,909 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:36,909 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:36,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,910 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:36,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,910 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,910 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-02 03:54:36,910 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:36,911 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-02 03:54:36,911 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:36,911 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-02 03:54:36,911 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:36,911 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:36,911 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:36,911 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-02 03:54:36,911 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. started... 2023-11-02 03:54:36,911 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c9354336ff3eea83ac372fc04a8d438e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:37,327 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201356e394ddb9d4c00b16efe36d13476a1 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201356e394ddb9d4c00b16efe36d13476a1 2023-11-02 03:54:37,732 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/d13820e398444212a8f5e20760f1ec13 2023-11-02 03:54:37,737 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/d13820e398444212a8f5e20760f1ec13 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/d13820e398444212a8f5e20760f1ec13 2023-11-02 03:54:37,740 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/d13820e398444212a8f5e20760f1ec13, entries=1, sequenceid=13, filesize=5.0 K 2023-11-02 03:54:37,741 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c9354336ff3eea83ac372fc04a8d438e in 830ms, sequenceid=13, compaction requested=true 2023-11-02 03:54:37,741 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:54:37,741 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:37,741 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:37,741 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:37,742 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:37,742 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-02 03:54:37,742 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-02 03:54:37,743 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:37,743 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:37,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:37,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:37,744 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,744 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:37,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:37,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:37,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:37,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:37,745 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-02 03:54:37,745 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e34e589[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:37,745 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-02 03:54:37,746 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,748 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,748 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-02 03:54:37,748 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-02 03:54:37,748 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:37,748 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:37,748 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:37,750 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:37,750 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-02 03:54:37,750 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:37,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:37,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:37,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:37,750 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-02 03:54:37,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:37,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:37,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:37,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:37,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:37,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:37,753 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-02 03:54:37,753 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-02 03:54:37,753 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:37,753 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:37,753 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,753 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:37,755 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,755 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:37,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:37,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,755 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:37,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:37,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:37,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:37,755 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:37,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:37,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:37,756 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:37,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:37,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:37,762 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:37,763 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,763 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:37,763 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:37,763 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:37,763 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:37,763 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:37,763 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:37,763 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:37,763 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:37,763 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,763 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:37,763 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,763 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:37,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:37,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:37,863 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:37,864 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:37,867 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:37,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:37,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,869 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:37,869 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:37,869 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-02 03:54:37,869 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-02 03:54:37,870 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,870 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,873 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:37,873 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:37,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:37,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:37,874 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:37,874 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:37,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,874 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:37,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,874 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,875 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-02 03:54:37,875 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:37,875 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-02 03:54:37,875 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:37,875 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-02 03:54:37,875 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:37,875 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:37,875 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:37,875 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. started... 2023-11-02 03:54:37,875 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c9354336ff3eea83ac372fc04a8d438e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:38,287 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120531ccc9a0db7a4bacab7717e35f1c96bd to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120531ccc9a0db7a4bacab7717e35f1c96bd 2023-11-02 03:54:38,693 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/ddea0f105b0e41998f259a9971cb4c82 2023-11-02 03:54:38,698 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/ddea0f105b0e41998f259a9971cb4c82 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/ddea0f105b0e41998f259a9971cb4c82 2023-11-02 03:54:38,702 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/ddea0f105b0e41998f259a9971cb4c82, entries=1, sequenceid=17, filesize=5.0 K 2023-11-02 03:54:38,724 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-02 03:54:38,724 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1e097ec5 to 127.0.0.1:65185 2023-11-02 03:54:38,724 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:54:38,724 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x30457c61 to 127.0.0.1:65185 2023-11-02 03:54:38,725 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:54:38,725 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-02 03:54:38,725 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6139e3b2 to 127.0.0.1:65185 2023-11-02 03:54:38,725 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:54:38,725 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-02 03:54:38,725 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x57bdaa3f to 127.0.0.1:65185 2023-11-02 03:54:38,725 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:54:38,726 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-02 03:54:38,726 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5a405a6f to 127.0.0.1:65185 2023-11-02 03:54:38,726 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:54:38,726 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c9354336ff3eea83ac372fc04a8d438e in 851ms, sequenceid=17, compaction requested=true 2023-11-02 03:54:38,726 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:54:38,726 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:38,726 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:38,727 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:38,727 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:38,727 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-02 03:54:38,727 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-02 03:54:38,731 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:38,731 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:38,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:38,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:38,731 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,731 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:38,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:38,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:38,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:38,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:38,733 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-02 03:54:38,733 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78eded65[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:38,733 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-02 03:54:38,733 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,734 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,734 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,734 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-02 03:54:38,734 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-02 03:54:38,734 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:38,734 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:38,734 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:38,735 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-02 03:54:38,735 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:38,735 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:38,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:38,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:38,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:38,736 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-02 03:54:38,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:38,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:38,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:38,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:38,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:38,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:38,738 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-02 03:54:38,738 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-02 03:54:38,738 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:38,738 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:38,738 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,738 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:38,741 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,741 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:38,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:38,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,741 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:38,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:38,741 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:38,741 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:38,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:38,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:38,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:38,743 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:38,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:38,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:38,745 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:38,745 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,745 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,745 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:38,745 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:38,745 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:38,745 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-11-02 03:54:38,745 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:38,745 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:38,745 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:38,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:38,746 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:38,745 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:38,746 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:38,745 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:38,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:38,746 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,846 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:38,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:38,850 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:38,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:38,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,852 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:38,852 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:38,852 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-02 03:54:38,852 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-02 03:54:38,852 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,852 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,853 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:38,853 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:38,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:38,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:38,854 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:38,854 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:38,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,854 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:38,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,854 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,855 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-02 03:54:38,855 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:38,855 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-02 03:54:38,855 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:38,855 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-02 03:54:38,855 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:38,855 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:38,855 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:38,855 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. started... 2023-11-02 03:54:38,855 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c9354336ff3eea83ac372fc04a8d438e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:39,266 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512289e930b8108774d4494462d0c5cb5eb17 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512289e930b8108774d4494462d0c5cb5eb17 2023-11-02 03:54:39,672 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/a54f9b2681534a95a2ce0e547bbba8f8 2023-11-02 03:54:39,676 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/a54f9b2681534a95a2ce0e547bbba8f8 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/a54f9b2681534a95a2ce0e547bbba8f8 2023-11-02 03:54:39,680 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/a54f9b2681534a95a2ce0e547bbba8f8, entries=1, sequenceid=21, filesize=5.0 K 2023-11-02 03:54:39,681 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c9354336ff3eea83ac372fc04a8d438e in 826ms, sequenceid=21, compaction requested=true 2023-11-02 03:54:39,681 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:54:39,681 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:39,681 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:39,681 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:39,681 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:39,681 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-02 03:54:39,681 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-02 03:54:39,683 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:39,683 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:39,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:39,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:39,683 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,683 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:39,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:39,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:39,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:39,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:39,684 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-02 03:54:39,685 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@22a721b5[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:39,685 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-02 03:54:39,685 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,687 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,687 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-02 03:54:39,687 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-02 03:54:39,687 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:39,687 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:39,687 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:39,688 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-02 03:54:39,688 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:39,688 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:39,688 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-02 03:54:39,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:39,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:39,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:39,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:39,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:39,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:39,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:39,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:39,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:39,691 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-02 03:54:39,691 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-02 03:54:39,691 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:39,691 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:39,691 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,691 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:39,692 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,692 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:39,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:39,692 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:39,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:39,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:39,693 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:39,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:39,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:39,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:39,694 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:39,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:39,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:39,696 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:39,696 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,696 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:39,696 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,696 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:39,696 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:39,697 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:39,696 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:39,696 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:39,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:39,697 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:39,697 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,697 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,697 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:39,697 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:39,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:39,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:39,797 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:39,798 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:39,801 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:39,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:39,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,803 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:39,803 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:39,803 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-02 03:54:39,803 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-02 03:54:39,803 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,804 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,809 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:39,809 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:39,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:39,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:39,810 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:39,810 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:39,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,810 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:39,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,810 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,811 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-02 03:54:39,811 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:39,811 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-02 03:54:39,811 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:39,811 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-02 03:54:39,811 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:39,811 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:39,811 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:39,811 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. started... 2023-11-02 03:54:39,811 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c9354336ff3eea83ac372fc04a8d438e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:40,222 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231cc847b2bf6964422bf5ee709d0be3caa to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231cc847b2bf6964422bf5ee709d0be3caa 2023-11-02 03:54:40,628 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/208bc677ecc3457fac7af93f9acbb362 2023-11-02 03:54:40,632 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/208bc677ecc3457fac7af93f9acbb362 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/208bc677ecc3457fac7af93f9acbb362 2023-11-02 03:54:40,636 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/208bc677ecc3457fac7af93f9acbb362, entries=1, sequenceid=25, filesize=5.0 K 2023-11-02 03:54:40,637 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c9354336ff3eea83ac372fc04a8d438e in 826ms, sequenceid=25, compaction requested=true 2023-11-02 03:54:40,637 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:54:40,637 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:40,637 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:40,637 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:40,637 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:40,637 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-02 03:54:40,637 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-02 03:54:40,638 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:40,639 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:40,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:40,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:40,639 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,639 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:40,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:40,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:40,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:40,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:40,640 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-02 03:54:40,640 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2293f449[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:40,640 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-02 03:54:40,640 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,641 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,642 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-02 03:54:40,642 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-02 03:54:40,642 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:40,642 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:40,642 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:40,643 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-02 03:54:40,643 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:40,643 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:40,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:40,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:40,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:40,643 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-02 03:54:40,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:40,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:40,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:40,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:40,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:40,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:40,646 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-02 03:54:40,646 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-02 03:54:40,646 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:40,646 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:40,646 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,646 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:40,648 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,648 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:40,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:40,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,648 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:40,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:40,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:40,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:40,649 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:40,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:40,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:40,650 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:40,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:40,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:40,652 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:40,652 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:40,652 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:40,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:40,652 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:40,653 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-11-02 03:54:40,653 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:40,653 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:40,653 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:40,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:40,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:40,652 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,653 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:40,653 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:40,653 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:40,653 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,653 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,653 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,753 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:40,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:40,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:40,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:40,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,759 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:40,759 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:40,759 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-02 03:54:40,759 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-02 03:54:40,760 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,760 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,761 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:40,761 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:40,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:40,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:40,761 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:40,761 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:40,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,762 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:40,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,762 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,762 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-02 03:54:40,762 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:40,762 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-02 03:54:40,762 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:40,762 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-02 03:54:40,762 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:40,763 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:40,763 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:40,763 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. started... 2023-11-02 03:54:40,763 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c9354336ff3eea83ac372fc04a8d438e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:41,174 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601013a1b1289812d40ee8cdc7cf9def06ea3 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601013a1b1289812d40ee8cdc7cf9def06ea3 2023-11-02 03:54:41,580 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/c6c141ac6f634c5893d17f3614e6f431 2023-11-02 03:54:41,584 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/c6c141ac6f634c5893d17f3614e6f431 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/c6c141ac6f634c5893d17f3614e6f431 2023-11-02 03:54:41,587 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/c6c141ac6f634c5893d17f3614e6f431, entries=1, sequenceid=29, filesize=5.0 K 2023-11-02 03:54:41,588 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c9354336ff3eea83ac372fc04a8d438e in 825ms, sequenceid=29, compaction requested=true 2023-11-02 03:54:41,588 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:54:41,588 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:41,588 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:41,588 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:41,588 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:41,588 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-02 03:54:41,588 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-02 03:54:41,590 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,590 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:41,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:41,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:41,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:41,590 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,590 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:41,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:41,592 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-02 03:54:41,592 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5bf2553f[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:41,592 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-02 03:54:41,592 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,593 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,593 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-02 03:54:41,593 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-02 03:54:41,593 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:41,593 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:41,593 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:41,594 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:41,594 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-02 03:54:41,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:41,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:41,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:41,594 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:41,595 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-02 03:54:41,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:41,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:41,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:41,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:41,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:41,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:41,597 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-02 03:54:41,597 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-02 03:54:41,597 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:41,597 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:41,597 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,597 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:41,599 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,599 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:41,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:41,599 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:41,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:41,599 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:41,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:41,599 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:41,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:41,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:41,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:41,601 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:41,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:41,603 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:41,603 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:41,603 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:41,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:41,603 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:41,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:41,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:41,603 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:41,603 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,603 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:41,603 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,603 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:41,603 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:41,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:41,603 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,603 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:41,704 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:41,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:41,708 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:41,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:41,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,710 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:41,710 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:41,710 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-02 03:54:41,710 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-02 03:54:41,710 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,710 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,711 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:41,711 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:41,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:41,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:41,712 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:41,712 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:41,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,712 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:41,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,712 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,713 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-02 03:54:41,713 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:41,713 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-02 03:54:41,713 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:41,713 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-02 03:54:41,724 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:41,724 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:41,724 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:41,724 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. started... 2023-11-02 03:54:41,724 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c9354336ff3eea83ac372fc04a8d438e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:42,136 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103cb782eaad81c40529ec632e0d5d9edf7 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103cb782eaad81c40529ec632e0d5d9edf7 2023-11-02 03:54:42,542 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/e93b887ded314513b8b8b396c0b7e155 2023-11-02 03:54:42,546 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/e93b887ded314513b8b8b396c0b7e155 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/e93b887ded314513b8b8b396c0b7e155 2023-11-02 03:54:42,549 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/e93b887ded314513b8b8b396c0b7e155, entries=1, sequenceid=33, filesize=5.0 K 2023-11-02 03:54:42,550 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c9354336ff3eea83ac372fc04a8d438e in 826ms, sequenceid=33, compaction requested=true 2023-11-02 03:54:42,550 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:54:42,550 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:42,550 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:42,550 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:42,550 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:42,550 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-02 03:54:42,550 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-02 03:54:42,552 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:42,552 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:42,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:42,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:42,552 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,552 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:42,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:42,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:42,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:42,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:42,554 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-02 03:54:42,554 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ce30de1[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:42,554 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-02 03:54:42,554 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,559 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,560 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-02 03:54:42,560 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-02 03:54:42,560 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:42,560 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:42,560 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:42,561 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:42,561 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-02 03:54:42,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:42,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:42,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:42,561 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:42,561 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-02 03:54:42,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:42,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:42,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:42,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:42,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:42,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:42,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-02 03:54:42,563 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-02 03:54:42,563 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:42,563 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:42,563 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,563 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:42,565 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,565 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:42,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:42,565 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:42,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:42,565 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:42,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:42,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:42,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:42,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:42,567 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:42,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:42,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:42,569 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:42,569 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:42,569 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:42,569 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-11-02 03:54:42,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:42,569 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:42,569 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:42,569 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:42,569 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,569 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:42,569 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:42,569 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,569 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:42,569 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:42,569 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,569 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:42,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:42,670 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:42,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:42,674 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:42,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:42,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,676 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:42,676 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:42,676 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-02 03:54:42,676 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-02 03:54:42,676 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,676 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,679 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:42,679 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:42,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:42,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:42,679 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:42,679 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:42,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,680 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:42,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,680 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,680 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-02 03:54:42,681 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:42,681 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-02 03:54:42,681 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:42,681 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-02 03:54:42,681 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:42,681 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:42,681 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:42,681 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. started... 2023-11-02 03:54:42,681 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c9354336ff3eea83ac372fc04a8d438e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:43,096 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090767fed9dd5f0c413fa63bf781b80acffe to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015090767fed9dd5f0c413fa63bf781b80acffe 2023-11-02 03:54:43,502 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/75b20189de7741a39905e436744b7c8c 2023-11-02 03:54:43,506 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/75b20189de7741a39905e436744b7c8c as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/75b20189de7741a39905e436744b7c8c 2023-11-02 03:54:43,510 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/75b20189de7741a39905e436744b7c8c, entries=1, sequenceid=37, filesize=5.0 K 2023-11-02 03:54:43,511 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c9354336ff3eea83ac372fc04a8d438e in 830ms, sequenceid=37, compaction requested=true 2023-11-02 03:54:43,511 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:54:43,511 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:43,511 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:43,511 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:43,511 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:43,511 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-02 03:54:43,511 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-02 03:54:43,513 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,513 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:43,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:43,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:43,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:43,513 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,513 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:43,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:43,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:43,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:43,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:43,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-02 03:54:43,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@57d22741[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:43,515 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-02 03:54:43,515 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,516 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,516 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-02 03:54:43,516 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-02 03:54:43,516 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:43,516 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:43,516 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:43,517 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:43,517 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-02 03:54:43,517 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:43,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:43,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:43,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:43,517 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-02 03:54:43,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:43,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:43,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:43,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:43,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:43,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:43,520 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-02 03:54:43,520 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-02 03:54:43,520 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:43,520 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:43,520 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,520 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:43,522 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,522 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:43,522 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:43,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:43,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:43,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:43,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:43,523 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:43,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:43,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:43,525 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:43,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:43,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:43,527 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:43,527 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,527 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:43,527 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,527 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-11-02 03:54:43,527 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:43,527 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:43,527 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:43,527 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:43,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:43,527 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:43,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:43,527 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,527 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:43,527 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:43,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:43,528 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,528 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:43,628 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:43,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:43,632 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:43,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:43,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,633 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:43,633 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:43,633 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-11-02 03:54:43,633 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-11-02 03:54:43,634 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,634 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,635 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:43,635 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:43,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:43,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:43,635 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:43,635 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:43,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:43,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,636 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,636 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-11-02 03:54:43,636 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:43,636 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-11-02 03:54:43,637 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:43,637 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-11-02 03:54:43,637 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:54:43,637 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:43,637 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:43,637 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. started... 2023-11-02 03:54:43,637 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c9354336ff3eea83ac372fc04a8d438e 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:54:44,048 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120a19023df10fb4a25949b2462bd09935b to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120a19023df10fb4a25949b2462bd09935b 2023-11-02 03:54:44,454 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/fc4f9e41fc9c4ddca3bf4edc3001f185 2023-11-02 03:54:44,458 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/fc4f9e41fc9c4ddca3bf4edc3001f185 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/fc4f9e41fc9c4ddca3bf4edc3001f185 2023-11-02 03:54:44,461 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/fc4f9e41fc9c4ddca3bf4edc3001f185, entries=1, sequenceid=41, filesize=5.0 K 2023-11-02 03:54:44,462 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for c9354336ff3eea83ac372fc04a8d438e in 825ms, sequenceid=41, compaction requested=true 2023-11-02 03:54:44,462 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:54:44,462 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:44,462 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:54:44,462 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:54:44,462 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:44,462 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-11-02 03:54:44,462 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-11-02 03:54:44,464 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,464 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:44,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:44,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:44,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:44,464 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,464 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:44,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:44,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:44,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:44,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:44,466 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-11-02 03:54:44,466 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6adeb04e[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:44,466 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-11-02 03:54:44,466 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,468 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,468 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-11-02 03:54:44,468 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-11-02 03:54:44,468 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:44,468 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:44,468 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:44,469 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:44,469 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-11-02 03:54:44,469 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:44,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:44,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:44,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:44,469 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-11-02 03:54:44,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:44,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:44,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:44,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:44,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:44,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:44,471 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-11-02 03:54:44,471 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-11-02 03:54:44,472 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:44,472 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:44,472 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,472 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:44,476 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,476 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:44,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:44,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,476 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:44,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:44,477 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:44,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:44,477 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:44,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:44,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:44,478 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:44,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:44,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:44,482 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:44,482 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,482 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:44,482 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:44,482 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:44,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:44,482 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:44,482 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:44,482 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:44,482 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:44,482 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,482 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,482 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,482 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:44,482 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:44,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:44,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:44,582 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:44,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:44,584 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41275] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:44,585 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] procedure2.ProcedureExecutor(1028): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-02 03:54:44,586 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-02 03:54:44,587 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:54:44,587 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698897884587 2023-11-02 03:54:44,589 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-02 03:54:44,590 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-11-02 03:54:44,590 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-02 03:54:44,591 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] zookeeper.ReadOnlyZKClient(139): Connect 0x3e76502f to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:54:44,595 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@18b89204, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:54:44,595 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:45,036 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010367fc7c07e8164ae78d743c4a1857d42b to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010367fc7c07e8164ae78d743c4a1857d42b 2023-11-02 03:54:45,037 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-02 03:54:45,050 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-02 03:54:45,051 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40006, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-02 03:54:45,058 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-02 03:54:45,059 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46620, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-02 03:54:45,068 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/11220823d7ae4fab909ddaaf389f1638 first=Optional[r3] last=Optional[r6] 2023-11-02 03:54:45,068 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/11220823d7ae4fab909ddaaf389f1638}] 2023-11-02 03:54:45,074 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/11220823d7ae4fab909ddaaf389f1638 for inclusion in c9354336ff3eea83ac372fc04a8d438e/family1 2023-11-02 03:54:45,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-11-02 03:54:45,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last= 2023-11-02 03:54:45,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:54:45,078 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/11220823d7ae4fab909ddaaf389f1638 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__dmp2a1uhqrmm2c5v2fr8kg0os8ouknv7g15dcsa0g07t08rdjcaspj4l8ekrvvec/family1/11220823d7ae4fab909ddaaf389f1638 2023-11-02 03:54:45,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__dmp2a1uhqrmm2c5v2fr8kg0os8ouknv7g15dcsa0g07t08rdjcaspj4l8ekrvvec/family1/11220823d7ae4fab909ddaaf389f1638 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/8826c69bf05046b4b10216486f96bd47_SeqId_44_ 2023-11-02 03:54:45,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__dmp2a1uhqrmm2c5v2fr8kg0os8ouknv7g15dcsa0g07t08rdjcaspj4l8ekrvvec/family1/11220823d7ae4fab909ddaaf389f1638 into c9354336ff3eea83ac372fc04a8d438e/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/8826c69bf05046b4b10216486f96bd47_SeqId_44_ - updating store file list. 2023-11-02 03:54:45,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/8826c69bf05046b4b10216486f96bd47_SeqId_44_ into c9354336ff3eea83ac372fc04a8d438e/family1 2023-11-02 03:54:45,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__dmp2a1uhqrmm2c5v2fr8kg0os8ouknv7g15dcsa0g07t08rdjcaspj4l8ekrvvec/family1/11220823d7ae4fab909ddaaf389f1638 into c9354336ff3eea83ac372fc04a8d438e/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/8826c69bf05046b4b10216486f96bd47_SeqId_44_) 2023-11-02 03:54:45,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__dmp2a1uhqrmm2c5v2fr8kg0os8ouknv7g15dcsa0g07t08rdjcaspj4l8ekrvvec/family1/11220823d7ae4fab909ddaaf389f1638 2023-11-02 03:54:45,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-02 03:54:45,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-02 03:54:45,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(1912): c9354336ff3eea83ac372fc04a8d438e/family1 is initiating minor compaction 2023-11-02 03:54:45,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@bcbb4d6; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:54:45,088 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c9354336ff3eea83ac372fc04a8d438e/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:45,088 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/43c2ee158cc2419a964e58f0b2ff5324, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/d13820e398444212a8f5e20760f1ec13, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/ddea0f105b0e41998f259a9971cb4c82, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/a54f9b2681534a95a2ce0e547bbba8f8, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/208bc677ecc3457fac7af93f9acbb362, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/c6c141ac6f634c5893d17f3614e6f431, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/e93b887ded314513b8b8b396c0b7e155, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/75b20189de7741a39905e436744b7c8c, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/fc4f9e41fc9c4ddca3bf4edc3001f185, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/8826c69bf05046b4b10216486f96bd47_SeqId_44_] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp, totalSize=50.3 K 2023-11-02 03:54:45,088 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 43c2ee158cc2419a964e58f0b2ff5324, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-02 03:54:45,088 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting d13820e398444212a8f5e20760f1ec13, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-02 03:54:45,088 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting ddea0f105b0e41998f259a9971cb4c82, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-02 03:54:45,088 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting a54f9b2681534a95a2ce0e547bbba8f8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-02 03:54:45,088 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 208bc677ecc3457fac7af93f9acbb362, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-02 03:54:45,089 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting c6c141ac6f634c5893d17f3614e6f431, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-02 03:54:45,089 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting e93b887ded314513b8b8b396c0b7e155, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-02 03:54:45,089 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 75b20189de7741a39905e436744b7c8c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-02 03:54:45,089 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting fc4f9e41fc9c4ddca3bf4edc3001f185, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-02 03:54:45,089 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 8826c69bf05046b4b10216486f96bd47_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-02 03:54:45,090 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-02 03:54:45,090 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:54:45,092 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512289e930b8108774d4494462d0c5cb5eb17 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512289e930b8108774d4494462d0c5cb5eb17 2023-11-02 03:54:45,093 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231cc847b2bf6964422bf5ee709d0be3caa to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231cc847b2bf6964422bf5ee709d0be3caa 2023-11-02 03:54:45,094 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601013a1b1289812d40ee8cdc7cf9def06ea3 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601013a1b1289812d40ee8cdc7cf9def06ea3 2023-11-02 03:54:45,095 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103cb782eaad81c40529ec632e0d5d9edf7 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103cb782eaad81c40529ec632e0d5d9edf7 2023-11-02 03:54:45,095 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-11-02 03:54:45,095 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-11-02 03:54:45,096 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:45,096 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:45,096 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:45,096 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:45,096 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:45,096 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:45,096 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:45,096 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:45,113 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] throttle.PressureAwareThroughputController(145): c9354336ff3eea83ac372fc04a8d438e#family1#compaction#72 average throughput is 1.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-02 03:54:45,929 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/3c68b3024be64228a87cd1fa255a4fc3 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/3c68b3024be64228a87cd1fa255a4fc3 2023-11-02 03:54:45,933 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512051a816a5b000b4eb498bfa161eddcbb6c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512051a816a5b000b4eb498bfa161eddcbb6c 2023-11-02 03:54:45,934 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in c9354336ff3eea83ac372fc04a8d438e/family1 of c9354336ff3eea83ac372fc04a8d438e into 3c68b3024be64228a87cd1fa255a4fc3(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-02 03:54:45,934 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-02 03:54:45,934 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:54:45,935 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1698897285088; duration=0sec 2023-11-02 03:54:45,935 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:54:45,963 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/43c2ee158cc2419a964e58f0b2ff5324, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/d13820e398444212a8f5e20760f1ec13, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/ddea0f105b0e41998f259a9971cb4c82, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/a54f9b2681534a95a2ce0e547bbba8f8, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/208bc677ecc3457fac7af93f9acbb362, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/c6c141ac6f634c5893d17f3614e6f431, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/e93b887ded314513b8b8b396c0b7e155, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/75b20189de7741a39905e436744b7c8c, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/fc4f9e41fc9c4ddca3bf4edc3001f185, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/8826c69bf05046b4b10216486f96bd47_SeqId_44_] to archive 2023-11-02 03:54:45,964 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:54:45,966 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/43c2ee158cc2419a964e58f0b2ff5324 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/43c2ee158cc2419a964e58f0b2ff5324 2023-11-02 03:54:45,967 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/d13820e398444212a8f5e20760f1ec13 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/d13820e398444212a8f5e20760f1ec13 2023-11-02 03:54:45,969 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/ddea0f105b0e41998f259a9971cb4c82 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/ddea0f105b0e41998f259a9971cb4c82 2023-11-02 03:54:45,971 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/a54f9b2681534a95a2ce0e547bbba8f8 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/a54f9b2681534a95a2ce0e547bbba8f8 2023-11-02 03:54:45,972 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/208bc677ecc3457fac7af93f9acbb362 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/208bc677ecc3457fac7af93f9acbb362 2023-11-02 03:54:45,972 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7a713f2229bb4a929eb8f15a6a1492bf first=Optional[r00] last=Optional[r2] 2023-11-02 03:54:45,973 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7a713f2229bb4a929eb8f15a6a1492bf}] 2023-11-02 03:54:45,973 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/c6c141ac6f634c5893d17f3614e6f431 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/c6c141ac6f634c5893d17f3614e6f431 2023-11-02 03:54:45,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7a713f2229bb4a929eb8f15a6a1492bf for inclusion in c9354336ff3eea83ac372fc04a8d438e/family1 2023-11-02 03:54:45,975 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/e93b887ded314513b8b8b396c0b7e155 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/e93b887ded314513b8b8b396c0b7e155 2023-11-02 03:54:45,976 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/75b20189de7741a39905e436744b7c8c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/75b20189de7741a39905e436744b7c8c 2023-11-02 03:54:45,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-11-02 03:54:45,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last= 2023-11-02 03:54:45,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:54:45,979 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/fc4f9e41fc9c4ddca3bf4edc3001f185 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/fc4f9e41fc9c4ddca3bf4edc3001f185 2023-11-02 03:54:45,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7a713f2229bb4a929eb8f15a6a1492bf to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__s4sv66a6hvvve2fr5mernll79none0unjpo1qnaa5v790cb994fem7jmse76tdsa/family1/7a713f2229bb4a929eb8f15a6a1492bf 2023-11-02 03:54:45,980 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/8826c69bf05046b4b10216486f96bd47_SeqId_44_ to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/8826c69bf05046b4b10216486f96bd47_SeqId_44_ 2023-11-02 03:54:45,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__s4sv66a6hvvve2fr5mernll79none0unjpo1qnaa5v790cb994fem7jmse76tdsa/family1/7a713f2229bb4a929eb8f15a6a1492bf as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/651ea114b4db4927aab477f7e423ebcc_SeqId_47_ 2023-11-02 03:54:45,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__s4sv66a6hvvve2fr5mernll79none0unjpo1qnaa5v790cb994fem7jmse76tdsa/family1/7a713f2229bb4a929eb8f15a6a1492bf into c9354336ff3eea83ac372fc04a8d438e/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/651ea114b4db4927aab477f7e423ebcc_SeqId_47_ - updating store file list. 2023-11-02 03:54:45,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/651ea114b4db4927aab477f7e423ebcc_SeqId_47_ into c9354336ff3eea83ac372fc04a8d438e/family1 2023-11-02 03:54:45,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__s4sv66a6hvvve2fr5mernll79none0unjpo1qnaa5v790cb994fem7jmse76tdsa/family1/7a713f2229bb4a929eb8f15a6a1492bf into c9354336ff3eea83ac372fc04a8d438e/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/651ea114b4db4927aab477f7e423ebcc_SeqId_47_) 2023-11-02 03:54:45,986 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__s4sv66a6hvvve2fr5mernll79none0unjpo1qnaa5v790cb994fem7jmse76tdsa/family1/7a713f2229bb4a929eb8f15a6a1492bf 2023-11-02 03:54:45,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-02 03:54:45,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-02 03:54:45,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(1912): c9354336ff3eea83ac372fc04a8d438e/family1 is initiating minor compaction (all files) 2023-11-02 03:54:45,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5aa72387; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:54:45,988 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c9354336ff3eea83ac372fc04a8d438e/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:45,988 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/7a36b2104aff481393b84eb66326621c, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/3c68b3024be64228a87cd1fa255a4fc3, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/651ea114b4db4927aab477f7e423ebcc_SeqId_47_] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp, totalSize=388.1 K 2023-11-02 03:54:45,988 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 7a36b2104aff481393b84eb66326621c, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878824741 2023-11-02 03:54:45,989 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 3c68b3024be64228a87cd1fa255a4fc3, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621224741 2023-11-02 03:54:45,989 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 651ea114b4db4927aab477f7e423ebcc_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878824741 2023-11-02 03:54:45,990 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-02 03:54:45,990 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:54:45,992 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130eaa689dcaed04cf5b605c8b4dec562f0 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130eaa689dcaed04cf5b605c8b4dec562f0 2023-11-02 03:54:45,993 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201356e394ddb9d4c00b16efe36d13476a1 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151201356e394ddb9d4c00b16efe36d13476a1 2023-11-02 03:54:45,994 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120531ccc9a0db7a4bacab7717e35f1c96bd to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120531ccc9a0db7a4bacab7717e35f1c96bd 2023-11-02 03:54:45,994 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-11-02 03:54:45,995 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-11-02 03:54:45,995 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:45,995 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-02 03:54:45,995 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:54:45,998 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-02 03:54:45,999 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.4130 sec 2023-11-02 03:54:46,013 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] throttle.PressureAwareThroughputController(145): c9354336ff3eea83ac372fc04a8d438e#family1#compaction#73 average throughput is 17.96 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-02 03:54:46,071 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-11-02 03:54:46,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:46,481 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-11-02 03:54:46,482 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-11-02 03:54:46,482 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-11-02 03:54:46,484 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-11-02 03:54:46,486 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=c9354336ff3eea83ac372fc04a8d438e, REOPEN/MOVE}] 2023-11-02 03:54:46,487 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=c9354336ff3eea83ac372fc04a8d438e, REOPEN/MOVE 2023-11-02 03:54:46,487 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=c9354336ff3eea83ac372fc04a8d438e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:46,488 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698897286487"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897286487"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897286487"}]},"ts":"1698897286487"} 2023-11-02 03:54:46,489 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure c9354336ff3eea83ac372fc04a8d438e, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:54:46,640 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:46,641 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:46,641 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c9354336ff3eea83ac372fc04a8d438e, disabling compactions & flushes 2023-11-02 03:54:46,641 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:47,232 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/.tmp/family1/4722ca805a2e4b1caabe2ec91137d9f0 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/4722ca805a2e4b1caabe2ec91137d9f0 2023-11-02 03:54:47,237 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in c9354336ff3eea83ac372fc04a8d438e/family1 of c9354336ff3eea83ac372fc04a8d438e into 4722ca805a2e4b1caabe2ec91137d9f0(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-02 03:54:47,237 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:54:47,237 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:47,237 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:47,237 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. after waiting 0 ms 2023-11-02 03:54:47,237 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:47,237 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1698897285988; duration=1sec 2023-11-02 03:54:47,237 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:54:47,240 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/7a36b2104aff481393b84eb66326621c, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/3c68b3024be64228a87cd1fa255a4fc3, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/651ea114b4db4927aab477f7e423ebcc_SeqId_47_] to archive 2023-11-02 03:54:47,241 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:54:47,242 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/7a36b2104aff481393b84eb66326621c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/7a36b2104aff481393b84eb66326621c 2023-11-02 03:54:47,243 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/3c68b3024be64228a87cd1fa255a4fc3 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/3c68b3024be64228a87cd1fa255a4fc3 2023-11-02 03:54:47,244 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/651ea114b4db4927aab477f7e423ebcc_SeqId_47_ to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/651ea114b4db4927aab477f7e423ebcc_SeqId_47_ 2023-11-02 03:54:47,248 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-11-02 03:54:47,249 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:47,249 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:54:47,249 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3510): Not adding moved region record: c9354336ff3eea83ac372fc04a8d438e to self. 2023-11-02 03:54:47,250 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:47,250 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=c9354336ff3eea83ac372fc04a8d438e, regionState=CLOSED 2023-11-02 03:54:47,251 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698897287250"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897287250"}]},"ts":"1698897287250"} 2023-11-02 03:54:47,253 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=62, resume processing ppid=61 2023-11-02 03:54:47,253 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure c9354336ff3eea83ac372fc04a8d438e, server=jenkins-hbase4.apache.org,36627,1698897201574 in 763 msec 2023-11-02 03:54:47,253 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=c9354336ff3eea83ac372fc04a8d438e, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=true 2023-11-02 03:54:47,404 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=c9354336ff3eea83ac372fc04a8d438e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:47,404 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698897287403"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897287403"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897287403"}]},"ts":"1698897287403"} 2023-11-02 03:54:47,405 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure c9354336ff3eea83ac372fc04a8d438e, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:54:47,556 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:47,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:47,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c9354336ff3eea83ac372fc04a8d438e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.', STARTKEY => '', ENDKEY => ''} 2023-11-02 03:54:47,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:47,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:47,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:47,559 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:47,561 INFO [StoreOpener-c9354336ff3eea83ac372fc04a8d438e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:47,561 DEBUG [StoreOpener-c9354336ff3eea83ac372fc04a8d438e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1 2023-11-02 03:54:47,561 DEBUG [StoreOpener-c9354336ff3eea83ac372fc04a8d438e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1 2023-11-02 03:54:47,562 INFO [StoreOpener-c9354336ff3eea83ac372fc04a8d438e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c9354336ff3eea83ac372fc04a8d438e columnFamilyName family1 2023-11-02 03:54:47,566 DEBUG [StoreOpener-c9354336ff3eea83ac372fc04a8d438e-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/4722ca805a2e4b1caabe2ec91137d9f0 2023-11-02 03:54:47,566 INFO [StoreOpener-c9354336ff3eea83ac372fc04a8d438e-1] regionserver.HStore(310): Store=c9354336ff3eea83ac372fc04a8d438e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:54:47,567 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:47,568 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:47,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:47,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c9354336ff3eea83ac372fc04a8d438e; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70078285, jitterRate=0.0442478209733963}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-02 03:54:47,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:54:47,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e., pid=63, masterSystemTime=1698897287556 2023-11-02 03:54:47,573 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:47,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:47,574 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=c9354336ff3eea83ac372fc04a8d438e, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:47,574 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698897287573"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897287573"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897287573"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897287573"}]},"ts":"1698897287573"} 2023-11-02 03:54:47,575 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=63, resume processing ppid=61 2023-11-02 03:54:47,576 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure c9354336ff3eea83ac372fc04a8d438e, server=jenkins-hbase4.apache.org,36627,1698897201574 in 170 msec 2023-11-02 03:54:47,577 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=61, resume processing ppid=60 2023-11-02 03:54:47,577 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=c9354336ff3eea83ac372fc04a8d438e, REOPEN/MOVE in 1.0900 sec 2023-11-02 03:54:47,579 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=60, resume processing ppid=59 2023-11-02 03:54:47,579 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.0930 sec 2023-11-02 03:54:47,580 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.5080 sec 2023-11-02 03:54:47,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=59 2023-11-02 03:54:47,582 INFO [Listener at localhost/43647] compactions.TestMobCompactor(278): alter status finished 2023-11-02 03:54:47,583 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41275] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:47,584 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] procedure2.ProcedureExecutor(1028): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-02 03:54:47,585 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-02 03:54:47,586 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:54:47,586 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698897887586 2023-11-02 03:54:47,586 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-02 03:54:47,587 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-11-02 03:54:47,587 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-02 03:54:47,588 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] zookeeper.ReadOnlyZKClient(139): Connect 0x0a06ff61 to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:54:47,597 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@617f1caa, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:54:47,597 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:47,597 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:47,597 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:47,597 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:47,597 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:48,420 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511286b0bad1b1e4746f7a811186ffdd1b811 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511286b0bad1b1e4746f7a811186ffdd1b811 2023-11-02 03:54:48,421 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-02 03:54:48,434 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-02 03:54:48,435 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37202, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-02 03:54:48,440 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-02 03:54:48,442 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41480, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-02 03:54:48,449 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/1f053c23ee234835b6052f74dc4e18d7 first=Optional[r05] last=Optional[r8] 2023-11-02 03:54:48,449 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=51 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/1f053c23ee234835b6052f74dc4e18d7}] 2023-11-02 03:54:48,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/1f053c23ee234835b6052f74dc4e18d7 for inclusion in c9354336ff3eea83ac372fc04a8d438e/family1 2023-11-02 03:54:48,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-11-02 03:54:48,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last= 2023-11-02 03:54:48,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:54:48,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/1f053c23ee234835b6052f74dc4e18d7 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__cd5ktvo1oqjnhmdqp5c1rqpvpk07rq52nsqdkuu5pumaro0svqqielfdljb13f2f/family1/1f053c23ee234835b6052f74dc4e18d7 2023-11-02 03:54:48,458 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__cd5ktvo1oqjnhmdqp5c1rqpvpk07rq52nsqdkuu5pumaro0svqqielfdljb13f2f/family1/1f053c23ee234835b6052f74dc4e18d7 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/5b04ff8592ae42b99803135db10252c3_SeqId_53_ 2023-11-02 03:54:48,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__cd5ktvo1oqjnhmdqp5c1rqpvpk07rq52nsqdkuu5pumaro0svqqielfdljb13f2f/family1/1f053c23ee234835b6052f74dc4e18d7 into c9354336ff3eea83ac372fc04a8d438e/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/5b04ff8592ae42b99803135db10252c3_SeqId_53_ - updating store file list. 2023-11-02 03:54:48,462 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/5b04ff8592ae42b99803135db10252c3_SeqId_53_ into c9354336ff3eea83ac372fc04a8d438e/family1 2023-11-02 03:54:48,462 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__cd5ktvo1oqjnhmdqp5c1rqpvpk07rq52nsqdkuu5pumaro0svqqielfdljb13f2f/family1/1f053c23ee234835b6052f74dc4e18d7 into c9354336ff3eea83ac372fc04a8d438e/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/5b04ff8592ae42b99803135db10252c3_SeqId_53_) 2023-11-02 03:54:48,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__cd5ktvo1oqjnhmdqp5c1rqpvpk07rq52nsqdkuu5pumaro0svqqielfdljb13f2f/family1/1f053c23ee234835b6052f74dc4e18d7 2023-11-02 03:54:48,464 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-02 03:54:48,465 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:54:48,466 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120a19023df10fb4a25949b2462bd09935b to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120a19023df10fb4a25949b2462bd09935b 2023-11-02 03:54:48,467 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128f09777a069b342878edffeb7c46f6e94 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128f09777a069b342878edffeb7c46f6e94 2023-11-02 03:54:48,467 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-11-02 03:54:48,467 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-11-02 03:54:48,468 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:48,468 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:48,468 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:48,468 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-02 03:54:48,468 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:54:48,469 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-02 03:54:48,470 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 885 msec 2023-11-02 03:54:48,512 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-11-02 03:54:48,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:48,920 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-11-02 03:54:48,921 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-11-02 03:54:48,921 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-11-02 03:54:48,923 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-11-02 03:54:48,925 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=c9354336ff3eea83ac372fc04a8d438e, REOPEN/MOVE}] 2023-11-02 03:54:48,925 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=c9354336ff3eea83ac372fc04a8d438e, REOPEN/MOVE 2023-11-02 03:54:48,926 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=c9354336ff3eea83ac372fc04a8d438e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:48,926 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698897288926"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897288926"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897288926"}]},"ts":"1698897288926"} 2023-11-02 03:54:48,927 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure c9354336ff3eea83ac372fc04a8d438e, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:54:49,078 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:49,078 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:49,078 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c9354336ff3eea83ac372fc04a8d438e, disabling compactions & flushes 2023-11-02 03:54:49,078 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:49,078 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:49,079 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. after waiting 0 ms 2023-11-02 03:54:49,079 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:49,084 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-11-02 03:54:49,085 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:49,085 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:54:49,085 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3510): Not adding moved region record: c9354336ff3eea83ac372fc04a8d438e to self. 2023-11-02 03:54:49,086 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:49,086 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=c9354336ff3eea83ac372fc04a8d438e, regionState=CLOSED 2023-11-02 03:54:49,086 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698897289086"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897289086"}]},"ts":"1698897289086"} 2023-11-02 03:54:49,088 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=68, resume processing ppid=67 2023-11-02 03:54:49,088 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure c9354336ff3eea83ac372fc04a8d438e, server=jenkins-hbase4.apache.org,36627,1698897201574 in 160 msec 2023-11-02 03:54:49,089 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=c9354336ff3eea83ac372fc04a8d438e, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=true 2023-11-02 03:54:49,239 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=c9354336ff3eea83ac372fc04a8d438e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:49,239 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698897289239"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897289239"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897289239"}]},"ts":"1698897289239"} 2023-11-02 03:54:49,241 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure c9354336ff3eea83ac372fc04a8d438e, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:54:49,392 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:49,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:49,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c9354336ff3eea83ac372fc04a8d438e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.', STARTKEY => '', ENDKEY => ''} 2023-11-02 03:54:49,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:49,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:49,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:49,396 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:49,397 INFO [StoreOpener-c9354336ff3eea83ac372fc04a8d438e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:49,397 DEBUG [StoreOpener-c9354336ff3eea83ac372fc04a8d438e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1 2023-11-02 03:54:49,397 DEBUG [StoreOpener-c9354336ff3eea83ac372fc04a8d438e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1 2023-11-02 03:54:49,398 INFO [StoreOpener-c9354336ff3eea83ac372fc04a8d438e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c9354336ff3eea83ac372fc04a8d438e columnFamilyName family1 2023-11-02 03:54:49,402 DEBUG [StoreOpener-c9354336ff3eea83ac372fc04a8d438e-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/4722ca805a2e4b1caabe2ec91137d9f0 2023-11-02 03:54:49,405 DEBUG [StoreOpener-c9354336ff3eea83ac372fc04a8d438e-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/family1/5b04ff8592ae42b99803135db10252c3_SeqId_53_ 2023-11-02 03:54:49,406 INFO [StoreOpener-c9354336ff3eea83ac372fc04a8d438e-1] regionserver.HStore(310): Store=c9354336ff3eea83ac372fc04a8d438e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:54:49,406 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:49,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:49,410 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:54:49,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c9354336ff3eea83ac372fc04a8d438e; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59580211, jitterRate=-0.11218567192554474}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-02 03:54:49,410 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:54:49,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e., pid=69, masterSystemTime=1698897289392 2023-11-02 03:54:49,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:49,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:54:49,412 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=c9354336ff3eea83ac372fc04a8d438e, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:49,412 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1698897289412"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897289412"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897289412"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897289412"}]},"ts":"1698897289412"} 2023-11-02 03:54:49,415 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=69, resume processing ppid=67 2023-11-02 03:54:49,415 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure c9354336ff3eea83ac372fc04a8d438e, server=jenkins-hbase4.apache.org,36627,1698897201574 in 172 msec 2023-11-02 03:54:49,416 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=67, resume processing ppid=66 2023-11-02 03:54:49,416 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=c9354336ff3eea83ac372fc04a8d438e, REOPEN/MOVE in 490 msec 2023-11-02 03:54:49,417 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=66, resume processing ppid=65 2023-11-02 03:54:49,418 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 493 msec 2023-11-02 03:54:49,419 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 906 msec 2023-11-02 03:54:49,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=65 2023-11-02 03:54:49,420 INFO [Listener at localhost/43647] compactions.TestMobCompactor(278): alter status finished 2023-11-02 03:54:49,421 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41275] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-11-02 03:54:49,422 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] procedure2.ProcedureExecutor(1028): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-02 03:54:49,423 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-02 03:54:49,423 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:54:49,423 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698897889423 2023-11-02 03:54:49,424 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-02 03:54:49,424 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-11-02 03:54:49,424 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-02 03:54:49,425 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] zookeeper.ReadOnlyZKClient(139): Connect 0x21988c03 to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:54:49,429 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@521f9fdc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:54:49,429 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:49,429 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:49,429 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:49,429 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:49,429 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:49,429 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:49,429 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:49,429 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-11-02 03:54:49,429 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-11-02 03:54:49,429 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:54:49,430 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-11-02 03:54:49,431 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 9 msec 2023-11-02 03:54:49,475 INFO [Listener at localhost/43647] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=243 (was 252), OpenFileDescriptor=647 (was 660), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=133 (was 175), ProcessCount=170 (was 170), AvailableMemoryMB=11150 (was 11213) 2023-11-02 03:54:49,486 INFO [Listener at localhost/43647] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=243, OpenFileDescriptor=647, MaxFileDescriptor=60000, SystemLoadAverage=133, ProcessCount=170, AvailableMemoryMB=11150 2023-11-02 03:54:49,489 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-11-02 03:54:49,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-11-02 03:54:49,491 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-11-02 03:54:49,491 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-11-02 03:54:49,491 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-02 03:54:49,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-02 03:54:49,492 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986 2023-11-02 03:54:49,493 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986 empty. 2023-11-02 03:54:49,493 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986 2023-11-02 03:54:49,493 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-11-02 03:54:49,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-02 03:54:49,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-02 03:54:49,900 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-11-02 03:54:49,901 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => e548a478a7c542966de3a4a335be0986, NAME => 'testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp 2023-11-02 03:54:50,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-02 03:54:50,307 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:50,307 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1604): Closing e548a478a7c542966de3a4a335be0986, disabling compactions & flushes 2023-11-02 03:54:50,307 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986. 2023-11-02 03:54:50,307 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986. 2023-11-02 03:54:50,307 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986. after waiting 0 ms 2023-11-02 03:54:50,307 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986. 2023-11-02 03:54:50,307 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986. 2023-11-02 03:54:50,307 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1558): Region close journal for e548a478a7c542966de3a4a335be0986: 2023-11-02 03:54:50,309 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-11-02 03:54:50,310 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1698897290310"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897290310"}]},"ts":"1698897290310"} 2023-11-02 03:54:50,311 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-02 03:54:50,312 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-02 03:54:50,312 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897290312"}]},"ts":"1698897290312"} 2023-11-02 03:54:50,313 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-11-02 03:54:50,315 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=e548a478a7c542966de3a4a335be0986, ASSIGN}] 2023-11-02 03:54:50,317 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=e548a478a7c542966de3a4a335be0986, ASSIGN 2023-11-02 03:54:50,317 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=e548a478a7c542966de3a4a335be0986, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=false 2023-11-02 03:54:50,468 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=e548a478a7c542966de3a4a335be0986, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:50,468 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1698897290468"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897290468"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897290468"}]},"ts":"1698897290468"} 2023-11-02 03:54:50,469 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure e548a478a7c542966de3a4a335be0986, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:54:50,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-02 03:54:50,621 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:50,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986. 2023-11-02 03:54:50,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e548a478a7c542966de3a4a335be0986, NAME => 'testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986.', STARTKEY => '', ENDKEY => ''} 2023-11-02 03:54:50,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-11-02 03:54:50,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-11-02 03:54:50,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction e548a478a7c542966de3a4a335be0986 2023-11-02 03:54:50,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:50,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e548a478a7c542966de3a4a335be0986 2023-11-02 03:54:50,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e548a478a7c542966de3a4a335be0986 2023-11-02 03:54:50,627 INFO [StoreOpener-e548a478a7c542966de3a4a335be0986-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region e548a478a7c542966de3a4a335be0986 2023-11-02 03:54:50,629 DEBUG [StoreOpener-e548a478a7c542966de3a4a335be0986-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1 2023-11-02 03:54:50,629 DEBUG [StoreOpener-e548a478a7c542966de3a4a335be0986-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1 2023-11-02 03:54:50,629 INFO [StoreOpener-e548a478a7c542966de3a4a335be0986-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e548a478a7c542966de3a4a335be0986 columnFamilyName f1 2023-11-02 03:54:50,630 INFO [StoreOpener-e548a478a7c542966de3a4a335be0986-1] regionserver.HStore(310): Store=e548a478a7c542966de3a4a335be0986/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:54:50,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986 2023-11-02 03:54:50,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986 2023-11-02 03:54:50,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e548a478a7c542966de3a4a335be0986 2023-11-02 03:54:50,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:54:50,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e548a478a7c542966de3a4a335be0986; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63258403, jitterRate=-0.05737634003162384}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-02 03:54:50,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e548a478a7c542966de3a4a335be0986: 2023-11-02 03:54:50,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986., pid=73, masterSystemTime=1698897290621 2023-11-02 03:54:50,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986. 2023-11-02 03:54:50,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986. 2023-11-02 03:54:50,639 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=e548a478a7c542966de3a4a335be0986, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:50,639 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1698897290639"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897290639"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897290639"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897290639"}]},"ts":"1698897290639"} 2023-11-02 03:54:50,641 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=73, resume processing ppid=72 2023-11-02 03:54:50,641 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure e548a478a7c542966de3a4a335be0986, server=jenkins-hbase4.apache.org,36627,1698897201574 in 171 msec 2023-11-02 03:54:50,642 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=72, resume processing ppid=71 2023-11-02 03:54:50,642 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=e548a478a7c542966de3a4a335be0986, ASSIGN in 326 msec 2023-11-02 03:54:50,643 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-02 03:54:50,643 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897290643"}]},"ts":"1698897290643"} 2023-11-02 03:54:50,644 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-11-02 03:54:50,646 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-11-02 03:54:50,647 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1580 sec 2023-11-02 03:54:51,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-11-02 03:54:51,596 INFO [Listener at localhost/43647] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-11-02 03:54:51,596 DEBUG [Listener at localhost/43647] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-11-02 03:54:51,598 INFO [Listener at localhost/43647] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-11-02 03:54:51,601 INFO [Listener at localhost/43647] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-11-02 03:54:51,601 INFO [Listener at localhost/43647] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-11-02 03:54:51,601 INFO [Listener at localhost/43647] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-11-02 03:54:51,605 INFO [Listener at localhost/43647] regionserver.HRegion(2745): Flushing e548a478a7c542966de3a4a335be0986 1/1 column families, dataSize=37 B heapSize=368 B 2023-11-02 03:54:52,016 INFO [Listener at localhost/43647] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023110210507364636b4ed6a3ea4d816162057a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023110210507364636b4ed6a3ea4d816162057a 2023-11-02 03:54:52,421 INFO [Listener at localhost/43647] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/.tmp/f1/894c527e89c1449699e799e4336c4f0e 2023-11-02 03:54:52,426 DEBUG [Listener at localhost/43647] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/.tmp/f1/894c527e89c1449699e799e4336c4f0e as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e 2023-11-02 03:54:52,430 INFO [Listener at localhost/43647] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e, entries=1, sequenceid=5, filesize=4.9 K 2023-11-02 03:54:52,431 INFO [Listener at localhost/43647] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for e548a478a7c542966de3a4a335be0986 in 826ms, sequenceid=5, compaction requested=false 2023-11-02 03:54:52,431 DEBUG [Listener at localhost/43647] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-11-02 03:54:52,431 DEBUG [Listener at localhost/43647] regionserver.HRegion(2446): Flush status journal for e548a478a7c542966de3a4a335be0986: 2023-11-02 03:54:52,433 INFO [Listener at localhost/43647] regionserver.HRegion(2745): Flushing e548a478a7c542966de3a4a335be0986 1/1 column families, dataSize=37 B heapSize=368 B 2023-11-02 03:54:52,783 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-02 03:54:52,844 INFO [Listener at localhost/43647] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231102bb4c2c9be7c34d408efc67c03f94eb30 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20231102bb4c2c9be7c34d408efc67c03f94eb30 2023-11-02 03:54:53,249 INFO [Listener at localhost/43647] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/.tmp/f1/e541cf0a66c3484bb05cc2b48c961501 2023-11-02 03:54:53,253 DEBUG [Listener at localhost/43647] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/.tmp/f1/e541cf0a66c3484bb05cc2b48c961501 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501 2023-11-02 03:54:53,257 INFO [Listener at localhost/43647] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501, entries=1, sequenceid=9, filesize=4.9 K 2023-11-02 03:54:53,258 INFO [Listener at localhost/43647] regionserver.HRegion(2948): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for e548a478a7c542966de3a4a335be0986 in 825ms, sequenceid=9, compaction requested=false 2023-11-02 03:54:53,258 DEBUG [Listener at localhost/43647] regionserver.HRegion(2446): Flush status journal for e548a478a7c542966de3a4a335be0986: 2023-11-02 03:54:53,259 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41275] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-11-02 03:54:53,260 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] procedure2.ProcedureExecutor(1028): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-11-02 03:54:53,261 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-11-02 03:54:53,261 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:54:53,261 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698897893261 2023-11-02 03:54:53,262 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-02 03:54:53,262 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-11-02 03:54:53,262 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-11-02 03:54:53,403 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-11-02 03:54:53,403 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-02 03:54:53,403 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-11-02 03:54:53,403 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3e76502f to 127.0.0.1:65185 2023-11-02 03:54:53,403 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] zookeeper.ReadOnlyZKClient(139): Connect 0x3047fe44 to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:54:53,403 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:54:53,403 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-02 03:54:53,403 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,404 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0a06ff61 to 127.0.0.1:65185 2023-11-02 03:54:53,404 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:54:53,404 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,404 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x21988c03 to 127.0.0.1:65185 2023-11-02 03:54:53,404 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:54:53,409 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1e8876ef, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:54:53,409 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231102 for table='testGetAfterCompaction' and column='f1' 2023-11-02 03:54:53,454 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,455 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,505 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,505 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,556 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,556 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,607 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,607 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,658 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,658 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,708 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,708 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,759 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,759 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,810 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,810 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,860 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,860 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,911 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,911 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,962 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:53,962 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,013 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,013 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,063 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,064 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,114 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,114 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,165 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,165 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,216 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,216 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,233 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023110212db5b7eed234614a2b9d5fccbadd658 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023110212db5b7eed234614a2b9d5fccbadd658 2023-11-02 03:54:54,234 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231102 table='testGetAfterCompaction' column='f1' 2023-11-02 03:54:54,253 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-02 03:54:54,255 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37204, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-02 03:54:54,261 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-02 03:54:54,263 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41494, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-02 03:54:54,267 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,267 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,275 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231102/f1/cd041385c0504fdb8bb3cb512f11b94f first=Optional[r1] last=Optional[r2] 2023-11-02 03:54:54,276 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row with hfile group [{f1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231102/f1/cd041385c0504fdb8bb3cb512f11b94f}] 2023-11-02 03:54:54,280 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231102/f1/cd041385c0504fdb8bb3cb512f11b94f for inclusion in e548a478a7c542966de3a4a335be0986/f1 2023-11-02 03:54:54,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-11-02 03:54:54,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last= 2023-11-02 03:54:54,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for e548a478a7c542966de3a4a335be0986: 2023-11-02 03:54:54,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231102/f1/cd041385c0504fdb8bb3cb512f11b94f to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testGetAfterCompaction__c8advc3urrvt4hbsp4b0aa1dsb8v29dsinn3okkg4nujsnbggnqd0u7omgi3p1o4/f1/cd041385c0504fdb8bb3cb512f11b94f 2023-11-02 03:54:54,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testGetAfterCompaction__c8advc3urrvt4hbsp4b0aa1dsb8v29dsinn3okkg4nujsnbggnqd0u7omgi3p1o4/f1/cd041385c0504fdb8bb3cb512f11b94f as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_ 2023-11-02 03:54:54,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testGetAfterCompaction__c8advc3urrvt4hbsp4b0aa1dsb8v29dsinn3okkg4nujsnbggnqd0u7omgi3p1o4/f1/cd041385c0504fdb8bb3cb512f11b94f into e548a478a7c542966de3a4a335be0986/f1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_ - updating store file list. 2023-11-02 03:54:54,289 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_ into e548a478a7c542966de3a4a335be0986/f1 2023-11-02 03:54:54,289 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testGetAfterCompaction__c8advc3urrvt4hbsp4b0aa1dsb8v29dsinn3okkg4nujsnbggnqd0u7omgi3p1o4/f1/cd041385c0504fdb8bb3cb512f11b94f into e548a478a7c542966de3a4a335be0986/f1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_) 2023-11-02 03:54:54,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testGetAfterCompaction__c8advc3urrvt4hbsp4b0aa1dsb8v29dsinn3okkg4nujsnbggnqd0u7omgi3p1o4/f1/cd041385c0504fdb8bb3cb512f11b94f 2023-11-02 03:54:54,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(1912): e548a478a7c542966de3a4a335be0986/f1 is initiating minor compaction 2023-11-02 03:54:54,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7daccc29; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:54:54,290 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of e548a478a7c542966de3a4a335be0986/f1 in testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986. 2023-11-02 03:54:54,291 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/.tmp, totalSize=10.0 K 2023-11-02 03:54:54,291 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting e541cf0a66c3484bb05cc2b48c961501, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-02 03:54:54,291 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 14f34ee9879c42ec83cca0ff63063744_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-02 03:54:54,292 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231102 table='testGetAfterCompaction' column='f1' 2023-11-02 03:54:54,293 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:54:54,295 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023110210507364636b4ed6a3ea4d816162057a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023110210507364636b4ed6a3ea4d816162057a 2023-11-02 03:54:54,296 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20231102bb4c2c9be7c34d408efc67c03f94eb30 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20231102bb4c2c9be7c34d408efc67c03f94eb30 2023-11-02 03:54:54,296 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-11-02 03:54:54,296 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20231102' 2023-11-02 03:54:54,296 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231102 for table='testGetAfterCompaction' and column='f1' 2023-11-02 03:54:54,296 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-11-02 03:54:54,297 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:54:54,298 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-11-02 03:54:54,299 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 1.0380 sec 2023-11-02 03:54:54,299 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] throttle.PressureAwareThroughputController(145): e548a478a7c542966de3a4a335be0986#f1#compaction#76 average throughput is 0.25 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-02 03:54:54,317 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698897294229; access_time=1698897293421; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-02 03:54:54,318 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,318 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,322 DEBUG [Listener at localhost/43647] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-11-02 03:54:54,325 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37212, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-11-02 03:54:54,325 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36627] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986. 2023-11-02 03:54:54,326 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36627] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-11-02 03:54:54,326 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36627] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-11-02 03:54:54,326 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36627] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-11-02 03:54:54,326 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=36627] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986. because compaction request was cancelled 2023-11-02 03:54:54,377 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698897294229; access_time=1698897293421; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-02 03:54:54,377 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,377 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,428 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698897294229; access_time=1698897293421; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-02 03:54:54,428 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,428 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,479 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698897294229; access_time=1698897293421; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-02 03:54:54,479 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,479 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,530 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698897294229; access_time=1698897293421; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-02 03:54:54,530 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,530 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,581 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698897294229; access_time=1698897293421; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-02 03:54:54,581 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,581 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,631 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698897294229; access_time=1698897293421; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-02 03:54:54,632 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,632 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,682 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698897294229; access_time=1698897293421; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-02 03:54:54,682 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,682 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,733 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698897294229; access_time=1698897293421; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-02 03:54:54,733 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,733 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,784 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698897294229; access_time=1698897293421; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-02 03:54:54,784 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,784 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,834 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698897294229; access_time=1698897293421; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-02 03:54:54,834 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,834 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,885 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698897294229; access_time=1698897293421; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-02 03:54:54,885 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,885 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,936 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698897294229; access_time=1698897293421; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-02 03:54:54,936 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,936 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,986 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698897294229; access_time=1698897293421; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-02 03:54:54,986 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:54,987 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:55,037 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698897294229; access_time=1698897293421; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-02 03:54:55,037 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:55,037 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:55,088 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698897294229; access_time=1698897293421; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-02 03:54:55,088 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:55,088 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:55,112 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/.tmp/f1/ec516cee55874001ac893e32af85d3b6 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/ec516cee55874001ac893e32af85d3b6 2023-11-02 03:54:55,116 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in e548a478a7c542966de3a4a335be0986/f1 of e548a478a7c542966de3a4a335be0986 into ec516cee55874001ac893e32af85d3b6(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-02 03:54:55,116 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for e548a478a7c542966de3a4a335be0986: 2023-11-02 03:54:55,116 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1698897294290; duration=0sec 2023-11-02 03:54:55,116 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:54:55,138 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698897294229; access_time=1698897293421; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-02 03:54:55,139 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:55,139 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:55,139 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/ec516cee55874001ac893e32af85d3b6; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1698897295108; access_time=1698897294296; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:55,189 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1698897294229; access_time=1698897293421; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-11-02 03:54:55,189 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:55,189 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897293249; access_time=1698897292435; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:55,189 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/ec516cee55874001ac893e32af85d3b6; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1698897295108; access_time=1698897294296; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:55,205 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_] to archive 2023-11-02 03:54:55,205 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:54:55,207 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/e541cf0a66c3484bb05cc2b48c961501 2023-11-02 03:54:55,208 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_ to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/14f34ee9879c42ec83cca0ff63063744_SeqId_12_ 2023-11-02 03:54:55,240 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1698897292421; access_time=1698897291607; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:55,240 INFO [Listener at localhost/43647] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/ec516cee55874001ac893e32af85d3b6; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1698897295108; access_time=1698897294296; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-11-02 03:54:55,355 INFO [Listener at localhost/43647] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-11-02 03:54:55,359 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-11-02 03:54:55,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-11-02 03:54:55,367 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897295367"}]},"ts":"1698897295367"} 2023-11-02 03:54:55,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=75 2023-11-02 03:54:55,368 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-11-02 03:54:55,371 INFO [PEWorker-5] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-11-02 03:54:55,372 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=e548a478a7c542966de3a4a335be0986, UNASSIGN}] 2023-11-02 03:54:55,373 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=e548a478a7c542966de3a4a335be0986, UNASSIGN 2023-11-02 03:54:55,374 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=e548a478a7c542966de3a4a335be0986, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:55,374 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1698897295374"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897295374"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897295374"}]},"ts":"1698897295374"} 2023-11-02 03:54:55,375 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure e548a478a7c542966de3a4a335be0986, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:54:55,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=75 2023-11-02 03:54:55,526 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:55,526 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close e548a478a7c542966de3a4a335be0986 2023-11-02 03:54:55,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e548a478a7c542966de3a4a335be0986, disabling compactions & flushes 2023-11-02 03:54:55,527 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986. 2023-11-02 03:54:55,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986. 2023-11-02 03:54:55,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986. after waiting 0 ms 2023-11-02 03:54:55,527 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986. 2023-11-02 03:54:55,533 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-11-02 03:54:55,533 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-11-02 03:54:55,534 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986. 2023-11-02 03:54:55,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e548a478a7c542966de3a4a335be0986: 2023-11-02 03:54:55,535 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed e548a478a7c542966de3a4a335be0986 2023-11-02 03:54:55,536 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=e548a478a7c542966de3a4a335be0986, regionState=CLOSED 2023-11-02 03:54:55,536 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1698897295536"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897295536"}]},"ts":"1698897295536"} 2023-11-02 03:54:55,538 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=77, resume processing ppid=76 2023-11-02 03:54:55,538 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure e548a478a7c542966de3a4a335be0986, server=jenkins-hbase4.apache.org,36627,1698897201574 in 162 msec 2023-11-02 03:54:55,539 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=76, resume processing ppid=75 2023-11-02 03:54:55,539 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=e548a478a7c542966de3a4a335be0986, UNASSIGN in 166 msec 2023-11-02 03:54:55,540 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897295540"}]},"ts":"1698897295540"} 2023-11-02 03:54:55,541 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-11-02 03:54:55,542 INFO [PEWorker-5] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-11-02 03:54:55,543 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 183 msec 2023-11-02 03:54:55,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=75 2023-11-02 03:54:55,670 INFO [Listener at localhost/43647] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-11-02 03:54:55,674 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-11-02 03:54:55,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-11-02 03:54:55,682 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-11-02 03:54:55,683 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-11-02 03:54:55,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=78 2023-11-02 03:54:55,686 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986 2023-11-02 03:54:55,687 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1, FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/recovered.edits] 2023-11-02 03:54:55,691 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/894c527e89c1449699e799e4336c4f0e 2023-11-02 03:54:55,692 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/ec516cee55874001ac893e32af85d3b6 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/f1/ec516cee55874001ac893e32af85d3b6 2023-11-02 03:54:55,694 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/recovered.edits/15.seqid to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986/recovered.edits/15.seqid 2023-11-02 03:54:55,694 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testGetAfterCompaction/e548a478a7c542966de3a4a335be0986 2023-11-02 03:54:55,695 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-11-02 03:54:55,695 DEBUG [PEWorker-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-11-02 03:54:55,696 DEBUG [PEWorker-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-11-02 03:54:55,698 DEBUG [PEWorker-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023110212db5b7eed234614a2b9d5fccbadd658 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e2023110212db5b7eed234614a2b9d5fccbadd658 2023-11-02 03:54:55,699 DEBUG [PEWorker-4] backup.HFileArchiver(596): Deleted hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-11-02 03:54:55,700 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-11-02 03:54:55,722 WARN [PEWorker-4] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-11-02 03:54:55,724 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-11-02 03:54:55,725 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-11-02 03:54:55,725 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-11-02 03:54:55,725 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1698897295725"}]},"ts":"9223372036854775807"} 2023-11-02 03:54:55,726 INFO [PEWorker-4] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-11-02 03:54:55,726 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => e548a478a7c542966de3a4a335be0986, NAME => 'testGetAfterCompaction,,1698897289489.e548a478a7c542966de3a4a335be0986.', STARTKEY => '', ENDKEY => ''}] 2023-11-02 03:54:55,726 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-11-02 03:54:55,727 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1698897295726"}]},"ts":"9223372036854775807"} 2023-11-02 03:54:55,730 INFO [PEWorker-4] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-11-02 03:54:55,732 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-11-02 03:54:55,732 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 57 msec 2023-11-02 03:54:55,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=78 2023-11-02 03:54:55,785 INFO [Listener at localhost/43647] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-11-02 03:54:55,797 INFO [Listener at localhost/43647] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=239 (was 243), OpenFileDescriptor=622 (was 647), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=123 (was 133), ProcessCount=170 (was 170), AvailableMemoryMB=11082 (was 11150) 2023-11-02 03:54:55,807 INFO [Listener at localhost/43647] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=239, OpenFileDescriptor=622, MaxFileDescriptor=60000, SystemLoadAverage=123, ProcessCount=170, AvailableMemoryMB=11081 2023-11-02 03:54:55,809 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-11-02 03:54:55,809 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-11-02 03:54:55,810 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-11-02 03:54:55,810 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-11-02 03:54:55,811 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-02 03:54:55,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-02 03:54:55,813 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:54:55,813 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:54:55,813 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c 2023-11-02 03:54:55,813 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3 empty. 2023-11-02 03:54:55,813 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c empty. 2023-11-02 03:54:55,814 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90 empty. 2023-11-02 03:54:55,814 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:54:55,814 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c 2023-11-02 03:54:55,814 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:54:55,814 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-11-02 03:54:55,912 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-02 03:54:56,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-02 03:54:56,224 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-11-02 03:54:56,225 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7675): creating {ENCODED => b478efa9db26260ecef7476a80a8478c, NAME => 'testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp 2023-11-02 03:54:56,226 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7675): creating {ENCODED => dd77f6ddad975771f984fe41bad1dd90, NAME => 'testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp 2023-11-02 03:54:56,226 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6ec4530a62d461a9c6949db814d37fd3, NAME => 'testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp 2023-11-02 03:54:56,243 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:56,243 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1604): Closing dd77f6ddad975771f984fe41bad1dd90, disabling compactions & flushes 2023-11-02 03:54:56,243 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:54:56,243 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:54:56,243 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. after waiting 0 ms 2023-11-02 03:54:56,243 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:54:56,243 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:54:56,243 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1558): Region close journal for dd77f6ddad975771f984fe41bad1dd90: 2023-11-02 03:54:56,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-02 03:54:56,633 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:56,633 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1604): Closing b478efa9db26260ecef7476a80a8478c, disabling compactions & flushes 2023-11-02 03:54:56,633 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:54:56,633 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:54:56,633 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. after waiting 0 ms 2023-11-02 03:54:56,633 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:54:56,633 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:54:56,633 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1558): Region close journal for b478efa9db26260ecef7476a80a8478c: 2023-11-02 03:54:56,639 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:56,639 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1604): Closing 6ec4530a62d461a9c6949db814d37fd3, disabling compactions & flushes 2023-11-02 03:54:56,639 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:54:56,639 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:54:56,639 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. after waiting 0 ms 2023-11-02 03:54:56,639 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:54:56,639 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:54:56,639 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1558): Region close journal for 6ec4530a62d461a9c6949db814d37fd3: 2023-11-02 03:54:56,642 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-11-02 03:54:56,643 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698897296642"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897296642"}]},"ts":"1698897296642"} 2023-11-02 03:54:56,643 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698897296642"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897296642"}]},"ts":"1698897296642"} 2023-11-02 03:54:56,643 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698897296642"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897296642"}]},"ts":"1698897296642"} 2023-11-02 03:54:56,644 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-11-02 03:54:56,645 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-02 03:54:56,645 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897296645"}]},"ts":"1698897296645"} 2023-11-02 03:54:56,646 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-11-02 03:54:56,651 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6ec4530a62d461a9c6949db814d37fd3, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd77f6ddad975771f984fe41bad1dd90, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b478efa9db26260ecef7476a80a8478c, ASSIGN}] 2023-11-02 03:54:56,652 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6ec4530a62d461a9c6949db814d37fd3, ASSIGN 2023-11-02 03:54:56,652 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd77f6ddad975771f984fe41bad1dd90, ASSIGN 2023-11-02 03:54:56,652 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b478efa9db26260ecef7476a80a8478c, ASSIGN 2023-11-02 03:54:56,653 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6ec4530a62d461a9c6949db814d37fd3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=false 2023-11-02 03:54:56,653 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd77f6ddad975771f984fe41bad1dd90, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=false 2023-11-02 03:54:56,653 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b478efa9db26260ecef7476a80a8478c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=false 2023-11-02 03:54:56,804 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=b478efa9db26260ecef7476a80a8478c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:56,804 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=6ec4530a62d461a9c6949db814d37fd3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:56,804 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=dd77f6ddad975771f984fe41bad1dd90, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:56,804 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698897296804"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897296804"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897296804"}]},"ts":"1698897296804"} 2023-11-02 03:54:56,804 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698897296804"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897296804"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897296804"}]},"ts":"1698897296804"} 2023-11-02 03:54:56,805 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698897296804"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897296804"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897296804"}]},"ts":"1698897296804"} 2023-11-02 03:54:56,806 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=83, ppid=82, state=RUNNABLE; OpenRegionProcedure b478efa9db26260ecef7476a80a8478c, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:54:56,806 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=84, ppid=80, state=RUNNABLE; OpenRegionProcedure 6ec4530a62d461a9c6949db814d37fd3, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:54:56,808 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=85, ppid=81, state=RUNNABLE; OpenRegionProcedure dd77f6ddad975771f984fe41bad1dd90, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:54:56,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-02 03:54:56,959 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:56,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:54:56,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b478efa9db26260ecef7476a80a8478c, NAME => 'testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c.', STARTKEY => '2', ENDKEY => ''} 2023-11-02 03:54:56,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink b478efa9db26260ecef7476a80a8478c 2023-11-02 03:54:56,962 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:56,962 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b478efa9db26260ecef7476a80a8478c 2023-11-02 03:54:56,962 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b478efa9db26260ecef7476a80a8478c 2023-11-02 03:54:56,963 INFO [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b478efa9db26260ecef7476a80a8478c 2023-11-02 03:54:56,964 DEBUG [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1 2023-11-02 03:54:56,964 DEBUG [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1 2023-11-02 03:54:56,964 INFO [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b478efa9db26260ecef7476a80a8478c columnFamilyName family1 2023-11-02 03:54:56,965 INFO [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] regionserver.HStore(310): Store=b478efa9db26260ecef7476a80a8478c/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:54:56,966 INFO [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region b478efa9db26260ecef7476a80a8478c 2023-11-02 03:54:56,967 DEBUG [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2 2023-11-02 03:54:56,967 DEBUG [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2 2023-11-02 03:54:56,967 INFO [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b478efa9db26260ecef7476a80a8478c columnFamilyName family2 2023-11-02 03:54:56,968 INFO [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] regionserver.HStore(310): Store=b478efa9db26260ecef7476a80a8478c/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:54:56,969 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c 2023-11-02 03:54:56,969 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c 2023-11-02 03:54:56,971 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-02 03:54:56,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b478efa9db26260ecef7476a80a8478c 2023-11-02 03:54:56,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:54:56,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b478efa9db26260ecef7476a80a8478c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65713869, jitterRate=-0.02078704535961151}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-02 03:54:56,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b478efa9db26260ecef7476a80a8478c: 2023-11-02 03:54:56,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c., pid=83, masterSystemTime=1698897296959 2023-11-02 03:54:56,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:54:56,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:54:56,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:54:56,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => dd77f6ddad975771f984fe41bad1dd90, NAME => 'testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90.', STARTKEY => '1', ENDKEY => '2'} 2023-11-02 03:54:56,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:54:56,976 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=b478efa9db26260ecef7476a80a8478c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:56,976 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:56,977 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:54:56,977 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:54:56,977 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698897296976"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897296976"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897296976"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897296976"}]},"ts":"1698897296976"} 2023-11-02 03:54:56,978 INFO [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:54:56,979 DEBUG [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1 2023-11-02 03:54:56,979 DEBUG [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1 2023-11-02 03:54:56,979 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=83, resume processing ppid=82 2023-11-02 03:54:56,979 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=83, ppid=82, state=SUCCESS; OpenRegionProcedure b478efa9db26260ecef7476a80a8478c, server=jenkins-hbase4.apache.org,36627,1698897201574 in 172 msec 2023-11-02 03:54:56,980 INFO [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dd77f6ddad975771f984fe41bad1dd90 columnFamilyName family1 2023-11-02 03:54:56,980 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b478efa9db26260ecef7476a80a8478c, ASSIGN in 328 msec 2023-11-02 03:54:56,980 INFO [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] regionserver.HStore(310): Store=dd77f6ddad975771f984fe41bad1dd90/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:54:56,981 INFO [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:54:56,982 DEBUG [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2 2023-11-02 03:54:56,982 DEBUG [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2 2023-11-02 03:54:56,982 INFO [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dd77f6ddad975771f984fe41bad1dd90 columnFamilyName family2 2023-11-02 03:54:56,983 INFO [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] regionserver.HStore(310): Store=dd77f6ddad975771f984fe41bad1dd90/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:54:56,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:54:56,985 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:54:56,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-02 03:54:56,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:54:56,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:54:56,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened dd77f6ddad975771f984fe41bad1dd90; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69575399, jitterRate=0.03675423562526703}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-02 03:54:56,989 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for dd77f6ddad975771f984fe41bad1dd90: 2023-11-02 03:54:56,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90., pid=85, masterSystemTime=1698897296959 2023-11-02 03:54:56,991 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:54:56,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:54:56,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:54:56,991 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6ec4530a62d461a9c6949db814d37fd3, NAME => 'testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3.', STARTKEY => '', ENDKEY => '1'} 2023-11-02 03:54:56,991 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=dd77f6ddad975771f984fe41bad1dd90, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:56,991 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:54:56,991 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:54:56,991 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698897296991"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897296991"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897296991"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897296991"}]},"ts":"1698897296991"} 2023-11-02 03:54:56,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:54:56,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:54:56,993 INFO [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:54:56,994 DEBUG [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1 2023-11-02 03:54:56,994 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=85, resume processing ppid=81 2023-11-02 03:54:56,994 DEBUG [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1 2023-11-02 03:54:56,994 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=85, ppid=81, state=SUCCESS; OpenRegionProcedure dd77f6ddad975771f984fe41bad1dd90, server=jenkins-hbase4.apache.org,36627,1698897201574 in 186 msec 2023-11-02 03:54:56,995 INFO [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ec4530a62d461a9c6949db814d37fd3 columnFamilyName family1 2023-11-02 03:54:56,995 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd77f6ddad975771f984fe41bad1dd90, ASSIGN in 346 msec 2023-11-02 03:54:56,996 INFO [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] regionserver.HStore(310): Store=6ec4530a62d461a9c6949db814d37fd3/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:54:56,996 INFO [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:54:56,997 DEBUG [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2 2023-11-02 03:54:56,997 DEBUG [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2 2023-11-02 03:54:56,997 INFO [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ec4530a62d461a9c6949db814d37fd3 columnFamilyName family2 2023-11-02 03:54:56,998 INFO [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] regionserver.HStore(310): Store=6ec4530a62d461a9c6949db814d37fd3/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:54:56,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:54:56,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:54:57,001 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-02 03:54:57,002 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:54:57,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:54:57,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6ec4530a62d461a9c6949db814d37fd3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64181521, jitterRate=-0.04362080991268158}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-02 03:54:57,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6ec4530a62d461a9c6949db814d37fd3: 2023-11-02 03:54:57,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3., pid=84, masterSystemTime=1698897296959 2023-11-02 03:54:57,006 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:54:57,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:54:57,006 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=6ec4530a62d461a9c6949db814d37fd3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:57,006 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698897297006"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897297006"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897297006"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897297006"}]},"ts":"1698897297006"} 2023-11-02 03:54:57,008 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=84, resume processing ppid=80 2023-11-02 03:54:57,008 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=84, ppid=80, state=SUCCESS; OpenRegionProcedure 6ec4530a62d461a9c6949db814d37fd3, server=jenkins-hbase4.apache.org,36627,1698897201574 in 201 msec 2023-11-02 03:54:57,009 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=80, resume processing ppid=79 2023-11-02 03:54:57,010 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6ec4530a62d461a9c6949db814d37fd3, ASSIGN in 359 msec 2023-11-02 03:54:57,010 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-02 03:54:57,010 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897297010"}]},"ts":"1698897297010"} 2023-11-02 03:54:57,011 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-11-02 03:54:57,013 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-11-02 03:54:57,014 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.2040 sec 2023-11-02 03:54:57,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-11-02 03:54:57,915 INFO [Listener at localhost/43647] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-11-02 03:54:57,922 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:54:57,923 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:54:57,924 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:54:57,927 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:57,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-11-02 03:54:57,929 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-11-02 03:54:57,929 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:57,929 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-11-02 03:54:57,929 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-11-02 03:54:57,929 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:54:57,929 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:54:57,931 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:57,931 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:57,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:57,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:57,932 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:57,932 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:57,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:54:57,932 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:54:57,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:57,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:54:57,933 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-11-02 03:54:57,933 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-11-02 03:54:57,933 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-11-02 03:54:57,933 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:57,933 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-11-02 03:54:57,933 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:54:57,933 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. started... 2023-11-02 03:54:57,934 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dd77f6ddad975771f984fe41bad1dd90 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-02 03:54:57,934 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:54:57,934 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-02 03:54:57,934 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:54:57,934 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:57,934 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. started... 2023-11-02 03:54:57,934 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. started... 2023-11-02 03:54:57,935 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 6ec4530a62d461a9c6949db814d37fd3 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-02 03:54:57,935 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing b478efa9db26260ecef7476a80a8478c 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-02 03:54:58,353 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311020e34f72f161545fe81dd972e46735ff4 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311020e34f72f161545fe81dd972e46735ff4 2023-11-02 03:54:58,353 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231102342f97ac7df94b6d87dfc0a98523bbb9 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231102342f97ac7df94b6d87dfc0a98523bbb9 2023-11-02 03:54:58,357 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311023beda44052714bfba055af33c67ecdf6 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311023beda44052714bfba055af33c67ecdf6 2023-11-02 03:54:58,362 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family1/b5e1291b61a04f90918b7f4640c18c61 2023-11-02 03:54:58,758 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family1/14f9f1322bc6452cadb436583a718478 2023-11-02 03:54:58,766 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family1/9ed21d911efc44148a847bf3f3b4cd9b 2023-11-02 03:54:58,776 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023110231ae017da0d549489f6bcf43217b8a22 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023110231ae017da0d549489f6bcf43217b8a22 2023-11-02 03:54:59,172 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231102e801ed4a3d4948cf8c154e0dc2f14b74 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231102e801ed4a3d4948cf8c154e0dc2f14b74 2023-11-02 03:54:59,179 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231102541edba285f246e29a687e91402c074b to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231102541edba285f246e29a687e91402c074b 2023-11-02 03:54:59,180 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family2/26692d5f405449c9bbebf7d285316b79 2023-11-02 03:54:59,184 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family1/b5e1291b61a04f90918b7f4640c18c61 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/b5e1291b61a04f90918b7f4640c18c61 2023-11-02 03:54:59,187 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/b5e1291b61a04f90918b7f4640c18c61, entries=4, sequenceid=5, filesize=5.4 K 2023-11-02 03:54:59,188 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family2/26692d5f405449c9bbebf7d285316b79 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/26692d5f405449c9bbebf7d285316b79 2023-11-02 03:54:59,191 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/26692d5f405449c9bbebf7d285316b79, entries=2, sequenceid=5, filesize=5.1 K 2023-11-02 03:54:59,192 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for dd77f6ddad975771f984fe41bad1dd90 in 1258ms, sequenceid=5, compaction requested=false 2023-11-02 03:54:59,192 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-11-02 03:54:59,192 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dd77f6ddad975771f984fe41bad1dd90: 2023-11-02 03:54:59,192 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:54:59,193 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-02 03:54:59,577 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family2/e86f23de8c964b3f8b559674fc99287f 2023-11-02 03:54:59,581 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family1/14f9f1322bc6452cadb436583a718478 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/14f9f1322bc6452cadb436583a718478 2023-11-02 03:54:59,583 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family2/d59a2530e24141fd86350985f46c5be0 2023-11-02 03:54:59,585 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/14f9f1322bc6452cadb436583a718478, entries=4, sequenceid=5, filesize=5.4 K 2023-11-02 03:54:59,586 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family2/e86f23de8c964b3f8b559674fc99287f as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/e86f23de8c964b3f8b559674fc99287f 2023-11-02 03:54:59,587 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family1/9ed21d911efc44148a847bf3f3b4cd9b as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/9ed21d911efc44148a847bf3f3b4cd9b 2023-11-02 03:54:59,589 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/e86f23de8c964b3f8b559674fc99287f, entries=2, sequenceid=5, filesize=5.1 K 2023-11-02 03:54:59,590 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 6ec4530a62d461a9c6949db814d37fd3 in 1655ms, sequenceid=5, compaction requested=false 2023-11-02 03:54:59,591 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 6ec4530a62d461a9c6949db814d37fd3: 2023-11-02 03:54:59,591 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:54:59,591 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-02 03:54:59,591 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/9ed21d911efc44148a847bf3f3b4cd9b, entries=4, sequenceid=5, filesize=5.4 K 2023-11-02 03:54:59,592 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family2/d59a2530e24141fd86350985f46c5be0 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/d59a2530e24141fd86350985f46c5be0 2023-11-02 03:54:59,595 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/d59a2530e24141fd86350985f46c5be0, entries=2, sequenceid=5, filesize=5.1 K 2023-11-02 03:54:59,596 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for b478efa9db26260ecef7476a80a8478c in 1661ms, sequenceid=5, compaction requested=false 2023-11-02 03:54:59,596 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for b478efa9db26260ecef7476a80a8478c: 2023-11-02 03:54:59,596 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:54:59,596 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-02 03:54:59,596 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-02 03:54:59,596 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:59,596 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-11-02 03:54:59,596 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-11-02 03:54:59,598 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:54:59,598 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:59,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:59,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:59,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:59,598 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:54:59,598 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:54:59,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:59,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:59,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:54:59,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:59,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:59,599 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-11-02 03:54:59,599 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@23cc5b87[Count = 0] remaining members to acquire global barrier 2023-11-02 03:54:59,599 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-11-02 03:54:59,600 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:54:59,601 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:54:59,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:54:59,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:54:59,601 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-11-02 03:54:59,601 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-11-02 03:54:59,601 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:54:59,601 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:59,601 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:54:59,602 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:59,602 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-11-02 03:54:59,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:59,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:59,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:59,602 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:59,603 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-11-02 03:54:59,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:59,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:59,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:54:59,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:59,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:59,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:54:59,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:59,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:54:59,605 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-11-02 03:54:59,605 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-11-02 03:54:59,605 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:54:59,605 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:54:59,605 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-11-02 03:54:59,605 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:54:59,607 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:54:59,607 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:54:59,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:54:59,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:54:59,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:54:59,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:54:59,607 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:59,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:54:59,608 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:59,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:54:59,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:59,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:59,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:54:59,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:54:59,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:54:59,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:54:59,609 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:59,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:59,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:54:59,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:54:59,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:59,611 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:59,611 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:54:59,611 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:54:59,611 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:59,611 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:54:59,611 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:59,611 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-11-02 03:54:59,611 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:54:59,611 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-11-02 03:54:59,611 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:54:59,611 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:54:59,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:59,611 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:54:59,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:59,611 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:54:59,612 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-11-02 03:54:59,612 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:54:59,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:54:59,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:54:59,712 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:54:59,712 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:54:59,713 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:54:59,714 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:54:59,714 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:54:59,716 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:54:59,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-11-02 03:54:59,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-11-02 03:54:59,718 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-11-02 03:54:59,718 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:59,718 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-11-02 03:54:59,718 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-11-02 03:54:59,719 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:54:59,719 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:54:59,720 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:59,720 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:54:59,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:54:59,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:54:59,721 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:54:59,721 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:54:59,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:54:59,721 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:54:59,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:54:59,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:54:59,721 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-11-02 03:54:59,722 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-11-02 03:54:59,722 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-11-02 03:54:59,722 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-11-02 03:54:59,722 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:54:59,722 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-11-02 03:54:59,722 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:54:59,722 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. started... 2023-11-02 03:54:59,722 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-02 03:54:59,722 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:54:59,722 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dd77f6ddad975771f984fe41bad1dd90 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-02 03:54:59,722 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:54:59,722 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:54:59,723 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. started... 2023-11-02 03:54:59,723 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. started... 2023-11-02 03:54:59,723 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing b478efa9db26260ecef7476a80a8478c 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-02 03:54:59,723 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 6ec4530a62d461a9c6949db814d37fd3 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-02 03:55:00,142 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311022554ea023fd24ffb84021913dc7f96c0 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311022554ea023fd24ffb84021913dc7f96c0 2023-11-02 03:55:00,142 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311026c16c5c49f724abdae7b6b32320f65ff to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311026c16c5c49f724abdae7b6b32320f65ff 2023-11-02 03:55:00,142 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023110210ea62b1fd9c46d38b55f31359a0d585 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023110210ea62b1fd9c46d38b55f31359a0d585 2023-11-02 03:55:00,553 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family1/6005881c4ace40b6b1e416da26faa3eb 2023-11-02 03:55:00,553 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family1/0830c54a581e4ef6bb871baea529ded2 2023-11-02 03:55:00,554 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family1/656ab245fd0743afa9d05799e1b639e2 2023-11-02 03:55:00,573 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023110218ec4ff8aa4243d4bf9a3153acdf7dff to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023110218ec4ff8aa4243d4bf9a3153acdf7dff 2023-11-02 03:55:00,972 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231102ddb80c75b59640e3bb5dc44794ee82ea to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231102ddb80c75b59640e3bb5dc44794ee82ea 2023-11-02 03:55:00,973 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023110299c01bb84a054f5c919cf9eaba97a698 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023110299c01bb84a054f5c919cf9eaba97a698 2023-11-02 03:55:00,978 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family2/27c8799b53a54db7ab92c9e8b744c603 2023-11-02 03:55:00,981 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family1/6005881c4ace40b6b1e416da26faa3eb as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/6005881c4ace40b6b1e416da26faa3eb 2023-11-02 03:55:00,984 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/6005881c4ace40b6b1e416da26faa3eb, entries=4, sequenceid=9, filesize=5.4 K 2023-11-02 03:55:00,985 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family2/27c8799b53a54db7ab92c9e8b744c603 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/27c8799b53a54db7ab92c9e8b744c603 2023-11-02 03:55:00,988 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/27c8799b53a54db7ab92c9e8b744c603, entries=2, sequenceid=9, filesize=5.1 K 2023-11-02 03:55:00,989 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for dd77f6ddad975771f984fe41bad1dd90 in 1267ms, sequenceid=9, compaction requested=false 2023-11-02 03:55:00,989 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dd77f6ddad975771f984fe41bad1dd90: 2023-11-02 03:55:00,989 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:55:00,989 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-02 03:55:01,035 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-02 03:55:01,378 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family2/c68ad7afbd894381b90008227dceb20a 2023-11-02 03:55:01,378 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family2/861b63499b404c75822e982990e9f0c2 2023-11-02 03:55:01,382 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family1/0830c54a581e4ef6bb871baea529ded2 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/0830c54a581e4ef6bb871baea529ded2 2023-11-02 03:55:01,382 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family1/656ab245fd0743afa9d05799e1b639e2 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/656ab245fd0743afa9d05799e1b639e2 2023-11-02 03:55:01,386 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/0830c54a581e4ef6bb871baea529ded2, entries=4, sequenceid=9, filesize=5.4 K 2023-11-02 03:55:01,386 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/656ab245fd0743afa9d05799e1b639e2, entries=4, sequenceid=9, filesize=5.4 K 2023-11-02 03:55:01,387 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family2/c68ad7afbd894381b90008227dceb20a as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/c68ad7afbd894381b90008227dceb20a 2023-11-02 03:55:01,387 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family2/861b63499b404c75822e982990e9f0c2 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/861b63499b404c75822e982990e9f0c2 2023-11-02 03:55:01,390 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/c68ad7afbd894381b90008227dceb20a, entries=2, sequenceid=9, filesize=5.1 K 2023-11-02 03:55:01,390 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/861b63499b404c75822e982990e9f0c2, entries=2, sequenceid=9, filesize=5.1 K 2023-11-02 03:55:01,391 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 6ec4530a62d461a9c6949db814d37fd3 in 1668ms, sequenceid=9, compaction requested=false 2023-11-02 03:55:01,391 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for b478efa9db26260ecef7476a80a8478c in 1668ms, sequenceid=9, compaction requested=false 2023-11-02 03:55:01,391 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 6ec4530a62d461a9c6949db814d37fd3: 2023-11-02 03:55:01,391 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for b478efa9db26260ecef7476a80a8478c: 2023-11-02 03:55:01,391 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:55:01,391 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:55:01,391 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-02 03:55:01,391 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-02 03:55:01,391 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-02 03:55:01,391 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:01,391 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-11-02 03:55:01,392 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-11-02 03:55:01,397 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:01,397 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:01,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:01,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:01,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:01,398 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:01,398 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:01,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:01,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:01,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:01,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:01,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:01,399 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-11-02 03:55:01,399 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7714c61a[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:01,399 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-11-02 03:55:01,399 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:01,401 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:01,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:01,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:01,401 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-11-02 03:55:01,401 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-11-02 03:55:01,401 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:01,401 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:01,401 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:01,402 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-11-02 03:55:01,402 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:01,402 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:01,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:01,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:01,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:01,403 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-11-02 03:55:01,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:01,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:01,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:01,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:01,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:01,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:01,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:01,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:01,405 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-11-02 03:55:01,405 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-11-02 03:55:01,405 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:01,405 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:01,405 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-11-02 03:55:01,405 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:01,406 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:01,406 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:01,406 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:01,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:01,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:01,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:01,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:01,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:01,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:01,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:01,407 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:01,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:01,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:01,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:01,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:01,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:01,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:01,408 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:01,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:01,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:01,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:01,412 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:01,412 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:55:01,412 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:01,412 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:55:01,412 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:01,412 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:01,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-11-02 03:55:01,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:01,412 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:01,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:01,412 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:01,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:01,412 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:01,412 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:01,412 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:01,412 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:01,413 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:01,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:01,513 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:01,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:01,515 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:01,515 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:01,515 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:01,517 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:01,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-11-02 03:55:01,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-11-02 03:55:01,519 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-11-02 03:55:01,519 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:01,519 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-11-02 03:55:01,519 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-11-02 03:55:01,520 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:01,520 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:55:01,521 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:01,521 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:01,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:01,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:01,521 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:01,521 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:01,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:55:01,522 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:01,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:01,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:55:01,522 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-11-02 03:55:01,522 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-11-02 03:55:01,522 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-11-02 03:55:01,523 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-11-02 03:55:01,523 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:01,523 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-11-02 03:55:01,523 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:55:01,523 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-02 03:55:01,523 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:01,523 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:55:01,523 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:55:01,523 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. started... 2023-11-02 03:55:01,523 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. started... 2023-11-02 03:55:01,523 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. started... 2023-11-02 03:55:01,523 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 6ec4530a62d461a9c6949db814d37fd3 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-02 03:55:01,523 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing b478efa9db26260ecef7476a80a8478c 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-02 03:55:01,523 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dd77f6ddad975771f984fe41bad1dd90 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-02 03:55:01,547 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231102e561bf0308514c75b8062bddf947eb1c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231102e561bf0308514c75b8062bddf947eb1c 2023-11-02 03:55:01,946 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311024ec670ce8f05437389ed33e14c71a7da to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311024ec670ce8f05437389ed33e14c71a7da 2023-11-02 03:55:01,946 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311022eecb97c58c94d5c88d39499e2c15dd6 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311022eecb97c58c94d5c88d39499e2c15dd6 2023-11-02 03:55:01,952 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family1/a1705e018d254143ae0583e020fae5bc 2023-11-02 03:55:02,353 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family1/8a84221b334a4e9687250d6733f5fa71 2023-11-02 03:55:02,353 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family1/203e58c390f74338b4434e5c6a543c55 2023-11-02 03:55:02,364 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231102d63c3f81e2214c1587fd5ec7bcdd427e to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231102d63c3f81e2214c1587fd5ec7bcdd427e 2023-11-02 03:55:02,368 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family2/1af22c20f93447fb9a8a6c49fdd642ff 2023-11-02 03:55:02,372 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family1/a1705e018d254143ae0583e020fae5bc as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/a1705e018d254143ae0583e020fae5bc 2023-11-02 03:55:02,375 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/a1705e018d254143ae0583e020fae5bc, entries=4, sequenceid=13, filesize=5.4 K 2023-11-02 03:55:02,376 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family2/1af22c20f93447fb9a8a6c49fdd642ff as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/1af22c20f93447fb9a8a6c49fdd642ff 2023-11-02 03:55:02,379 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/1af22c20f93447fb9a8a6c49fdd642ff, entries=2, sequenceid=13, filesize=5.1 K 2023-11-02 03:55:02,379 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 6ec4530a62d461a9c6949db814d37fd3 in 856ms, sequenceid=13, compaction requested=true 2023-11-02 03:55:02,379 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 6ec4530a62d461a9c6949db814d37fd3: 2023-11-02 03:55:02,380 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:55:02,380 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-02 03:55:02,767 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311025d35ed5ec5934294baaab5f677a57e8a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311025d35ed5ec5934294baaab5f677a57e8a 2023-11-02 03:55:02,770 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231102c641bcec015b4c3da7f4e1d9141d5cb7 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231102c641bcec015b4c3da7f4e1d9141d5cb7 2023-11-02 03:55:03,172 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family2/f850896837b04a078722c50a1c779311 2023-11-02 03:55:03,175 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family2/f1a29a576b394c5394ced5cb7815b190 2023-11-02 03:55:03,176 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family1/203e58c390f74338b4434e5c6a543c55 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/203e58c390f74338b4434e5c6a543c55 2023-11-02 03:55:03,178 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family1/8a84221b334a4e9687250d6733f5fa71 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/8a84221b334a4e9687250d6733f5fa71 2023-11-02 03:55:03,180 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/203e58c390f74338b4434e5c6a543c55, entries=4, sequenceid=13, filesize=5.4 K 2023-11-02 03:55:03,181 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family2/f850896837b04a078722c50a1c779311 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/f850896837b04a078722c50a1c779311 2023-11-02 03:55:03,182 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/8a84221b334a4e9687250d6733f5fa71, entries=4, sequenceid=13, filesize=5.4 K 2023-11-02 03:55:03,183 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family2/f1a29a576b394c5394ced5cb7815b190 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/f1a29a576b394c5394ced5cb7815b190 2023-11-02 03:55:03,186 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/f850896837b04a078722c50a1c779311, entries=2, sequenceid=13, filesize=5.1 K 2023-11-02 03:55:03,187 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for b478efa9db26260ecef7476a80a8478c in 1664ms, sequenceid=13, compaction requested=true 2023-11-02 03:55:03,187 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for b478efa9db26260ecef7476a80a8478c: 2023-11-02 03:55:03,187 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:55:03,187 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-02 03:55:03,188 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/f1a29a576b394c5394ced5cb7815b190, entries=2, sequenceid=13, filesize=5.1 K 2023-11-02 03:55:03,189 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for dd77f6ddad975771f984fe41bad1dd90 in 1666ms, sequenceid=13, compaction requested=true 2023-11-02 03:55:03,189 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dd77f6ddad975771f984fe41bad1dd90: 2023-11-02 03:55:03,189 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:55:03,189 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-02 03:55:03,189 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-02 03:55:03,189 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:03,189 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-11-02 03:55:03,189 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-11-02 03:55:03,192 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:03,192 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:03,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:03,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:03,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:03,192 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:03,192 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:03,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:03,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:03,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:03,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:03,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:03,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-11-02 03:55:03,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31ba1f96[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:03,194 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-11-02 03:55:03,194 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:03,195 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:03,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:03,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:03,195 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-11-02 03:55:03,195 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-11-02 03:55:03,195 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:03,195 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:03,195 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:03,197 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-11-02 03:55:03,197 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:03,197 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:03,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:03,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:03,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:03,197 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-11-02 03:55:03,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:03,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:03,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:03,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:03,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:03,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:03,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:03,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:03,199 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-11-02 03:55:03,199 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-11-02 03:55:03,199 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:03,199 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:03,199 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-11-02 03:55:03,199 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:03,201 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:03,201 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:03,201 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:03,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:03,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:03,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:03,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:03,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:03,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:03,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:03,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:03,201 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:03,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:03,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:03,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:03,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:03,203 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:03,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:03,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:03,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:03,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:03,206 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:03,206 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:55:03,206 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:03,206 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:55:03,206 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-11-02 03:55:03,206 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-11-02 03:55:03,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:03,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:03,206 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:03,206 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:03,206 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:03,206 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:03,206 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:03,206 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:03,206 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:03,206 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:03,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:03,206 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:03,206 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:03,307 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:03,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:03,308 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:03,309 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:03,309 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:03,311 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:03,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-11-02 03:55:03,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-11-02 03:55:03,313 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-11-02 03:55:03,313 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:03,313 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-11-02 03:55:03,313 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-11-02 03:55:03,313 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:03,313 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:55:03,314 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:03,314 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:03,315 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:03,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:03,315 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:03,315 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:03,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:55:03,315 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:03,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:03,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:55:03,316 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-11-02 03:55:03,316 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-11-02 03:55:03,316 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-11-02 03:55:03,316 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-11-02 03:55:03,316 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:03,316 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-11-02 03:55:03,321 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:55:03,321 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. started... 2023-11-02 03:55:03,321 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:55:03,321 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-02 03:55:03,321 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dd77f6ddad975771f984fe41bad1dd90 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-02 03:55:03,321 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:55:03,321 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:03,321 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. started... 2023-11-02 03:55:03,321 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. started... 2023-11-02 03:55:03,321 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 6ec4530a62d461a9c6949db814d37fd3 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-02 03:55:03,321 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing b478efa9db26260ecef7476a80a8478c 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-02 03:55:03,335 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311020b8bff03978d4222a97f41a288302e66 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311020b8bff03978d4222a97f41a288302e66 2023-11-02 03:55:03,336 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311021353ebc9c5414069887b922ea4aaf9c0 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311021353ebc9c5414069887b922ea4aaf9c0 2023-11-02 03:55:03,735 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311021cc42c22257a40eb9c5aeb2784b3ad66 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311021cc42c22257a40eb9c5aeb2784b3ad66 2023-11-02 03:55:03,742 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family1/b36e505a69df415fa703d2406810ee79 2023-11-02 03:55:03,743 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family1/b1ddcc0e1aa7400bb8a5255f06753d0a 2023-11-02 03:55:04,140 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family1/161f3f15b77e44a4952ab837e081d9fa 2023-11-02 03:55:04,162 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023110264d619ba95e0487f9c5d1cb23cabc3c8 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023110264d619ba95e0487f9c5d1cb23cabc3c8 2023-11-02 03:55:04,162 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231102cee98bd818574201b98024dea9e0da98 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231102cee98bd818574201b98024dea9e0da98 2023-11-02 03:55:04,553 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311026b3cc1b9116d44cb91aa244c6aeb83ca to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311026b3cc1b9116d44cb91aa244c6aeb83ca 2023-11-02 03:55:04,566 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family2/cd29ec0b5df94911b3d52331056aa7eb 2023-11-02 03:55:04,567 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family2/bf61b264b68a4a5dacd4f9a9803c0106 2023-11-02 03:55:04,571 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family1/b36e505a69df415fa703d2406810ee79 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/b36e505a69df415fa703d2406810ee79 2023-11-02 03:55:04,571 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family1/b1ddcc0e1aa7400bb8a5255f06753d0a as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/b1ddcc0e1aa7400bb8a5255f06753d0a 2023-11-02 03:55:04,574 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/b1ddcc0e1aa7400bb8a5255f06753d0a, entries=4, sequenceid=17, filesize=5.4 K 2023-11-02 03:55:04,574 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/b36e505a69df415fa703d2406810ee79, entries=4, sequenceid=17, filesize=5.4 K 2023-11-02 03:55:04,575 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family2/bf61b264b68a4a5dacd4f9a9803c0106 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/bf61b264b68a4a5dacd4f9a9803c0106 2023-11-02 03:55:04,575 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family2/cd29ec0b5df94911b3d52331056aa7eb as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/cd29ec0b5df94911b3d52331056aa7eb 2023-11-02 03:55:04,579 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/bf61b264b68a4a5dacd4f9a9803c0106, entries=2, sequenceid=17, filesize=5.1 K 2023-11-02 03:55:04,579 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/cd29ec0b5df94911b3d52331056aa7eb, entries=2, sequenceid=17, filesize=5.1 K 2023-11-02 03:55:04,580 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 6ec4530a62d461a9c6949db814d37fd3 in 1259ms, sequenceid=17, compaction requested=true 2023-11-02 03:55:04,580 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for b478efa9db26260ecef7476a80a8478c in 1259ms, sequenceid=17, compaction requested=true 2023-11-02 03:55:04,580 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 6ec4530a62d461a9c6949db814d37fd3: 2023-11-02 03:55:04,580 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for b478efa9db26260ecef7476a80a8478c: 2023-11-02 03:55:04,580 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:55:04,580 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:55:04,580 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-02 03:55:04,580 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-02 03:55:04,958 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family2/0236f03dce144cd980710add026ae931 2023-11-02 03:55:04,962 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family1/161f3f15b77e44a4952ab837e081d9fa as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/161f3f15b77e44a4952ab837e081d9fa 2023-11-02 03:55:04,965 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/161f3f15b77e44a4952ab837e081d9fa, entries=4, sequenceid=17, filesize=5.4 K 2023-11-02 03:55:04,966 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family2/0236f03dce144cd980710add026ae931 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/0236f03dce144cd980710add026ae931 2023-11-02 03:55:04,969 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/0236f03dce144cd980710add026ae931, entries=2, sequenceid=17, filesize=5.1 K 2023-11-02 03:55:04,970 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for dd77f6ddad975771f984fe41bad1dd90 in 1649ms, sequenceid=17, compaction requested=true 2023-11-02 03:55:04,970 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dd77f6ddad975771f984fe41bad1dd90: 2023-11-02 03:55:04,970 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:55:04,970 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-02 03:55:04,970 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-02 03:55:04,970 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:04,970 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-11-02 03:55:04,970 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-11-02 03:55:04,972 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:04,972 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:04,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:04,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:04,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:04,972 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:04,972 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:04,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:04,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:04,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:04,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:04,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:04,973 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-11-02 03:55:04,974 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2ef9952f[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:04,974 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-11-02 03:55:04,974 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:04,975 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:04,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:04,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:04,975 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-11-02 03:55:04,975 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-11-02 03:55:04,975 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:04,975 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:04,975 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:04,977 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:04,977 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-11-02 03:55:04,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:04,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:04,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:04,977 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:04,978 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-11-02 03:55:04,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:04,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:04,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:04,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:04,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:04,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:04,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:04,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:04,980 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-11-02 03:55:04,980 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:04,980 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:04,980 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-11-02 03:55:04,980 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:04,980 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-11-02 03:55:04,981 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:04,981 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:04,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:04,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:04,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:04,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:04,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:04,981 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:04,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:04,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:04,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:04,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:04,982 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:04,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:04,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:04,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:04,984 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:04,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:04,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:04,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:04,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:04,986 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:04,986 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:04,986 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:55:04,986 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:04,986 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:55:04,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-11-02 03:55:04,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:04,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:04,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:04,986 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:04,987 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-11-02 03:55:04,987 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:04,986 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:04,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:04,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:04,987 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:04,987 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:04,987 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:04,987 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:05,087 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:05,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:05,098 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1583): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1698897305088 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-02 03:55:05,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1698897305098 (current time:1698897305098). 2023-11-02 03:55:05,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-11-02 03:55:05,100 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] zookeeper.ReadOnlyZKClient(139): Connect 0x1e940cc7 to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:55:05,105 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1da39aba, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:55:05,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-02 03:55:05,112 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52066, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-02 03:55:05,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1e940cc7 to 127.0.0.1:65185 2023-11-02 03:55:05,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:55:05,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] snapshot.SnapshotManager(637): No existing snapshot, attempting snapshot... 2023-11-02 03:55:05,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] snapshot.SnapshotManager(684): Table enabled, starting distributed snapshots for { ss=snaptb-1698897305088 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-02 03:55:05,121 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] snapshot.SnapshotManager(519): Unable to retrieve ACL status for path: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.hbase-snapshot, current working dir path: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.hbase-snapshot/.tmp/snaptb-1698897305088 org.apache.hadoop.hdfs.protocol.AclException: The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false. at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.checkAclsConfigFlag(FSDirAclOp.java:197) at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.getAclStatus(FSDirAclOp.java:150) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:6744) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getAclStatus(NameNodeRpcServer.java:2065) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getAclStatus(ClientNamenodeProtocolServerSideTranslatorPB.java:1378) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.getAclStatus(DFSClient.java:2736) at org.apache.hadoop.hdfs.DistributedFileSystem$51.doCall(DistributedFileSystem.java:2294) at org.apache.hadoop.hdfs.DistributedFileSystem$51.doCall(DistributedFileSystem.java:2291) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getAclStatus(DistributedFileSystem.java:2291) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.updateWorkingDirAclsIfRequired(SnapshotManager.java:517) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.prepareToTakeSnapshot(SnapshotManager.java:490) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.snapshotEnabledTable(SnapshotManager.java:559) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshotInternal(SnapshotManager.java:687) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshot(SnapshotManager.java:623) at org.apache.hadoop.hbase.master.MasterRpcServices.snapshot(MasterRpcServices.java:1588) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.AclException): The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false. at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.checkAclsConfigFlag(FSDirAclOp.java:197) at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.getAclStatus(FSDirAclOp.java:150) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:6744) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getAclStatus(NameNodeRpcServer.java:2065) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getAclStatus(ClientNamenodeProtocolServerSideTranslatorPB.java:1378) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.getAclStatus(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getAclStatus(ClientNamenodeProtocolTranslatorPB.java:1385) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getAclStatus(DFSClient.java:2734) ... 15 more 2023-11-02 03:55:05,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-02 03:55:05,129 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-02 03:55:05,130 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1698897905130 2023-11-02 03:55:05,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] snapshot.SnapshotManager(689): Started snapshot: { ss=snaptb-1698897305088 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-02 03:55:05,130 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1698897305088 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-11-02 03:55:05,131 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-02 03:55:05,132 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1028): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-11-02 03:55:05,132 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 3 msec 2023-11-02 03:55:05,133 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-11-02 03:55:05,133 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1698897905133 2023-11-02 03:55:05,135 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1698897305088 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:05,135 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-11-02 03:55:05,235 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-02 03:55:05,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1698897305088 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-02 03:55:05,239 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] snapshot.SnapshotManager(406): Snapshoting '{ ss=snaptb-1698897305088 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-02 03:55:05,239 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-11-02 03:55:05,439 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-02 03:55:05,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1698897305088 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-02 03:55:05,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] snapshot.SnapshotManager(406): Snapshoting '{ ss=snaptb-1698897305088 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-02 03:55:05,440 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-11-02 03:55:05,544 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1698897305088 2023-11-02 03:55:05,545 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1698897305088' 2023-11-02 03:55:05,545 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-11-02 03:55:05,545 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1698897305088' starting 'acquire' 2023-11-02 03:55:05,545 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1698897305088', kicking off acquire phase on members. 2023-11-02 03:55:05,545 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1698897305088 2023-11-02 03:55:05,545 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1698897305088 2023-11-02 03:55:05,547 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-11-02 03:55:05,547 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1698897305088/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:05,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-11-02 03:55:05,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-11-02 03:55:05,548 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1698897305088/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:05,548 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:05,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1698897305088 2023-11-02 03:55:05,548 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1698897305088 2023-11-02 03:55:05,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-11-02 03:55:05,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1698897305088 2023-11-02 03:55:05,549 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1698897305088 from table testCompactionWithHFileLink type FLUSH 2023-11-02 03:55:05,551 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1698897305088 2023-11-02 03:55:05,551 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1698897305088' with timeout 300000ms 2023-11-02 03:55:05,551 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-11-02 03:55:05,551 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1698897305088' starting 'acquire' stage 2023-11-02 03:55:05,551 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1698897305088' locally acquired 2023-11-02 03:55:05,551 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (snaptb-1698897305088) in zk 2023-11-02 03:55:05,552 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1698897305088/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:05,552 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1698897305088 2023-11-02 03:55:05,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1698897305088/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:05,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:05,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-11-02 03:55:05,553 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1698897305088 2023-11-02 03:55:05,553 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1698897305088' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:05,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:05,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:05,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1698897305088 2023-11-02 03:55:05,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:05,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:05,554 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'snaptb-1698897305088' on coordinator 2023-11-02 03:55:05,554 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f28f314[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:05,554 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1698897305088' starting 'in-barrier' execution. 2023-11-02 03:55:05,554 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1698897305088 2023-11-02 03:55:05,555 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1698897305088 2023-11-02 03:55:05,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1698897305088 2023-11-02 03:55:05,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1698897305088 2023-11-02 03:55:05,555 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1698897305088' received 'reached' from coordinator. 2023-11-02 03:55:05,555 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1698897305088/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:05,555 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:05,556 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:55:05,556 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:55:05,556 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. started... 2023-11-02 03:55:05,556 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. started... 2023-11-02 03:55:05,557 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for 6ec4530a62d461a9c6949db814d37fd3: 2023-11-02 03:55:05,557 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for dd77f6ddad975771f984fe41bad1dd90: 2023-11-02 03:55:05,557 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-11-02 03:55:05,557 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:55:05,557 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-11-02 03:55:05,557 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. started... 2023-11-02 03:55:05,557 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for b478efa9db26260ecef7476a80a8478c: 2023-11-02 03:55:05,557 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3.' region-info for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,557 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c.' region-info for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,557 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90.' region-info for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,562 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-11-02 03:55:05,562 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-11-02 03:55:05,562 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-11-02 03:55:05,566 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/9ed21d911efc44148a847bf3f3b4cd9b, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/656ab245fd0743afa9d05799e1b639e2, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/203e58c390f74338b4434e5c6a543c55, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/b36e505a69df415fa703d2406810ee79] hfiles 2023-11-02 03:55:05,566 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/14f9f1322bc6452cadb436583a718478, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/0830c54a581e4ef6bb871baea529ded2, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/a1705e018d254143ae0583e020fae5bc, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/b1ddcc0e1aa7400bb8a5255f06753d0a] hfiles 2023-11-02 03:55:05,566 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/9ed21d911efc44148a847bf3f3b4cd9b for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,566 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/b5e1291b61a04f90918b7f4640c18c61, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/6005881c4ace40b6b1e416da26faa3eb, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/8a84221b334a4e9687250d6733f5fa71, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/161f3f15b77e44a4952ab837e081d9fa] hfiles 2023-11-02 03:55:05,566 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/14f9f1322bc6452cadb436583a718478 for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,566 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/b5e1291b61a04f90918b7f4640c18c61 for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,569 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/0830c54a581e4ef6bb871baea529ded2 for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,569 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/656ab245fd0743afa9d05799e1b639e2 for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,569 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/6005881c4ace40b6b1e416da26faa3eb for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,569 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/203e58c390f74338b4434e5c6a543c55 for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,569 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/a1705e018d254143ae0583e020fae5bc for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,570 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/b36e505a69df415fa703d2406810ee79 for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,569 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/8a84221b334a4e9687250d6733f5fa71 for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,570 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/b1ddcc0e1aa7400bb8a5255f06753d0a for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,570 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/161f3f15b77e44a4952ab837e081d9fa for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,570 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/e86f23de8c964b3f8b559674fc99287f, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/c68ad7afbd894381b90008227dceb20a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/1af22c20f93447fb9a8a6c49fdd642ff, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/bf61b264b68a4a5dacd4f9a9803c0106] hfiles 2023-11-02 03:55:05,570 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/26692d5f405449c9bbebf7d285316b79, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/27c8799b53a54db7ab92c9e8b744c603, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/f1a29a576b394c5394ced5cb7815b190, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/0236f03dce144cd980710add026ae931] hfiles 2023-11-02 03:55:05,570 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/26692d5f405449c9bbebf7d285316b79 for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,570 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/e86f23de8c964b3f8b559674fc99287f for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,570 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/27c8799b53a54db7ab92c9e8b744c603 for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,570 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/f1a29a576b394c5394ced5cb7815b190 for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,570 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/d59a2530e24141fd86350985f46c5be0, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/861b63499b404c75822e982990e9f0c2, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/f850896837b04a078722c50a1c779311, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/cd29ec0b5df94911b3d52331056aa7eb] hfiles 2023-11-02 03:55:05,570 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/0236f03dce144cd980710add026ae931 for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,570 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/c68ad7afbd894381b90008227dceb20a for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,570 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/d59a2530e24141fd86350985f46c5be0 for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,570 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/1af22c20f93447fb9a8a6c49fdd642ff for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,570 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/861b63499b404c75822e982990e9f0c2 for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,570 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/bf61b264b68a4a5dacd4f9a9803c0106 for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,570 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/f850896837b04a078722c50a1c779311 for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,570 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/cd29ec0b5df94911b3d52331056aa7eb for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,740 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-02 03:55:05,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1698897305088 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-02 03:55:05,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] snapshot.SnapshotManager(406): Snapshoting '{ ss=snaptb-1698897305088 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-02 03:55:05,741 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-11-02 03:55:05,976 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. completed. 2023-11-02 03:55:05,976 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:55:05,976 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-11-02 03:55:05,982 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. completed. 2023-11-02 03:55:05,982 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:55:05,982 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. completed. 2023-11-02 03:55:05,982 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:55:05,982 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-11-02 03:55:05,982 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-11-02 03:55:05,982 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-11-02 03:55:05,982 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:05,982 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1698897305088' locally completed 2023-11-02 03:55:05,982 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1698897305088' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:05,985 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1698897305088' has notified controller of completion 2023-11-02 03:55:05,985 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1698897305088/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:05,985 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:05,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1698897305088/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:05,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:05,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-11-02 03:55:05,985 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1698897305088' completed. 2023-11-02 03:55:05,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:05,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:05,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1698897305088 2023-11-02 03:55:05,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:05,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:05,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1698897305088 2023-11-02 03:55:05,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:05,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1698897305088' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:05,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'snaptb-1698897305088', counting down latch. Waiting for 0 more 2023-11-02 03:55:05,987 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1698897305088' execution completed 2023-11-02 03:55:05,987 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:05,987 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:05,987 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1698897305088 2023-11-02 03:55:05,987 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1698897305088including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-11-02 03:55:05,989 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1698897305088 2023-11-02 03:55:05,989 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1698897305088 2023-11-02 03:55:05,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1698897305088 2023-11-02 03:55:05,989 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-11-02 03:55:05,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:05,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-11-02 03:55:05,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1698897305088 2023-11-02 03:55:05,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1698897305088 2023-11-02 03:55:05,989 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-11-02 03:55:05,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:05,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-11-02 03:55:05,989 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1698897305088/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:05,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1698897305088 2023-11-02 03:55:05,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1698897305088 2023-11-02 03:55:05,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:05,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1698897305088 2023-11-02 03:55:05,991 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1698897305088/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:05,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:05,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:05,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1698897305088 2023-11-02 03:55:05,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:05,993 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1698897305088/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:05,993 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-11-02 03:55:05,993 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1698897305088 2023-11-02 03:55:05,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-11-02 03:55:05,993 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:05,993 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/snaptb-1698897305088 because node does not exist (not an error) 2023-11-02 03:55:05,993 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1698897305088 2023-11-02 03:55:05,993 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-11-02 03:55:05,993 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1698897305088/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:05,993 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1698897305088 2023-11-02 03:55:05,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-11-02 03:55:05,993 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1698897305088 2023-11-02 03:55:05,993 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-11-02 03:55:05,993 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1698897305088 2023-11-02 03:55:05,993 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1698897305088 2023-11-02 03:55:05,993 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1698897305088 2023-11-02 03:55:05,994 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-11-02 03:55:05,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-11-02 03:55:05,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-11-02 03:55:05,995 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311021cc42c22257a40eb9c5aeb2784b3ad66, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231102342f97ac7df94b6d87dfc0a98523bbb9, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311024ec670ce8f05437389ed33e14c71a7da, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311026c16c5c49f724abdae7b6b32320f65ff, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311020b8bff03978d4222a97f41a288302e66, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023110210ea62b1fd9c46d38b55f31359a0d585, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311022eecb97c58c94d5c88d39499e2c15dd6, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311023beda44052714bfba055af33c67ecdf6, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311020e34f72f161545fe81dd972e46735ff4, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311021353ebc9c5414069887b922ea4aaf9c0, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311022554ea023fd24ffb84021913dc7f96c0, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231102e561bf0308514c75b8062bddf947eb1c] mob files 2023-11-02 03:55:05,995 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311021cc42c22257a40eb9c5aeb2784b3ad66 2023-11-02 03:55:05,995 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231102342f97ac7df94b6d87dfc0a98523bbb9 2023-11-02 03:55:05,995 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311024ec670ce8f05437389ed33e14c71a7da 2023-11-02 03:55:05,995 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311026c16c5c49f724abdae7b6b32320f65ff 2023-11-02 03:55:05,996 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311020b8bff03978d4222a97f41a288302e66 2023-11-02 03:55:05,996 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023110210ea62b1fd9c46d38b55f31359a0d585 2023-11-02 03:55:05,996 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311022eecb97c58c94d5c88d39499e2c15dd6 2023-11-02 03:55:05,996 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311023beda44052714bfba055af33c67ecdf6 2023-11-02 03:55:05,996 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311020e34f72f161545fe81dd972e46735ff4 2023-11-02 03:55:05,996 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311021353ebc9c5414069887b922ea4aaf9c0 2023-11-02 03:55:05,996 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311022554ea023fd24ffb84021913dc7f96c0 2023-11-02 03:55:05,996 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231102e561bf0308514c75b8062bddf947eb1c 2023-11-02 03:55:05,997 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023110218ec4ff8aa4243d4bf9a3153acdf7dff, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023110231ae017da0d549489f6bcf43217b8a22, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311026b3cc1b9116d44cb91aa244c6aeb83ca, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231102c641bcec015b4c3da7f4e1d9141d5cb7, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231102541edba285f246e29a687e91402c074b, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311025d35ed5ec5934294baaab5f677a57e8a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023110264d619ba95e0487f9c5d1cb23cabc3c8, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231102ddb80c75b59640e3bb5dc44794ee82ea, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023110299c01bb84a054f5c919cf9eaba97a698, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231102cee98bd818574201b98024dea9e0da98, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231102d63c3f81e2214c1587fd5ec7bcdd427e, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231102e801ed4a3d4948cf8c154e0dc2f14b74] mob files 2023-11-02 03:55:05,997 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023110218ec4ff8aa4243d4bf9a3153acdf7dff 2023-11-02 03:55:05,997 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023110231ae017da0d549489f6bcf43217b8a22 2023-11-02 03:55:05,997 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311026b3cc1b9116d44cb91aa244c6aeb83ca 2023-11-02 03:55:05,997 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231102c641bcec015b4c3da7f4e1d9141d5cb7 2023-11-02 03:55:05,998 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231102541edba285f246e29a687e91402c074b 2023-11-02 03:55:05,998 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311025d35ed5ec5934294baaab5f677a57e8a 2023-11-02 03:55:05,998 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023110264d619ba95e0487f9c5d1cb23cabc3c8 2023-11-02 03:55:05,998 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231102ddb80c75b59640e3bb5dc44794ee82ea 2023-11-02 03:55:05,998 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023110299c01bb84a054f5c919cf9eaba97a698 2023-11-02 03:55:05,998 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231102cee98bd818574201b98024dea9e0da98 2023-11-02 03:55:05,998 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231102d63c3f81e2214c1587fd5ec7bcdd427e 2023-11-02 03:55:05,998 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231102e801ed4a3d4948cf8c154e0dc2f14b74 2023-11-02 03:55:06,241 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-02 03:55:06,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1698897305088 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-02 03:55:06,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] snapshot.SnapshotManager(406): Snapshoting '{ ss=snaptb-1698897305088 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-02 03:55:06,242 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-11-02 03:55:06,403 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1698897305088 2023-11-02 03:55:06,405 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.hbase-snapshot/.tmp/snaptb-1698897305088 2023-11-02 03:55:06,847 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.hbase-snapshot/.tmp/snaptb-1698897305088 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.hbase-snapshot/snaptb-1698897305088 2023-11-02 03:55:06,847 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1698897305088 of table testCompactionWithHFileLink completed 2023-11-02 03:55:06,847 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.hbase-snapshot/.tmp/snaptb-1698897305088 2023-11-02 03:55:06,848 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.hbase-snapshot/.tmp/snaptb-1698897305088 2023-11-02 03:55:06,848 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1698897305088 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1698897305130Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1698897306403 (+1273 ms)Consolidate snapshot: snaptb-1698897305088 at 1698897306403Loading Region manifests for snaptb-1698897305088 at 1698897306403Writing data manifest for snaptb-1698897305088 at 1698897306412 (+9 ms)Verifying snapshot: snaptb-1698897305088 at 1698897306820 (+408 ms)Snapshot snaptb-1698897305088 of table testCompactionWithHFileLink completed at 1698897306847 (+27 ms) 2023-11-02 03:55:06,849 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-11-02 03:55:06,850 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 1.7180 sec 2023-11-02 03:55:07,243 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-02 03:55:07,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=snaptb-1698897305088 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-02 03:55:07,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] snapshot.SnapshotManager(403): Snapshot '{ ss=snaptb-1698897305088 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-11-02 03:55:07,272 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:07,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-11-02 03:55:07,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-11-02 03:55:07,274 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-11-02 03:55:07,274 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:07,274 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-11-02 03:55:07,274 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-11-02 03:55:07,275 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:07,275 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:55:07,276 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:07,276 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:07,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:07,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:07,276 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:07,276 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:07,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:55:07,276 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:07,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:07,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:55:07,277 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-11-02 03:55:07,277 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-11-02 03:55:07,277 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-11-02 03:55:07,277 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-11-02 03:55:07,277 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:07,277 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-11-02 03:55:07,277 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:55:07,277 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. started... 2023-11-02 03:55:07,277 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-02 03:55:07,278 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:55:07,278 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:55:07,278 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing dd77f6ddad975771f984fe41bad1dd90 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-02 03:55:07,278 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:07,278 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. started... 2023-11-02 03:55:07,278 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. started... 2023-11-02 03:55:07,278 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing b478efa9db26260ecef7476a80a8478c 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-02 03:55:07,278 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 6ec4530a62d461a9c6949db814d37fd3 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-02 03:55:07,700 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family1/5c3c49de4d8e4b148811e3352da5de23 2023-11-02 03:55:07,702 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family1/ea19fcf8ee40499c82c683b52a03a666 2023-11-02 03:55:07,705 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5c3c49de4d8e4b148811e3352da5de23 2023-11-02 03:55:07,706 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ea19fcf8ee40499c82c683b52a03a666 2023-11-02 03:55:08,096 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family1/9577dd05d75c4a509caaaf87ca707753 2023-11-02 03:55:08,100 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9577dd05d75c4a509caaaf87ca707753 2023-11-02 03:55:08,516 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family2/5d2a6c58378a49fcb1a09b66b8f07fc4 2023-11-02 03:55:08,519 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5d2a6c58378a49fcb1a09b66b8f07fc4 2023-11-02 03:55:08,520 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family1/5c3c49de4d8e4b148811e3352da5de23 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/5c3c49de4d8e4b148811e3352da5de23 2023-11-02 03:55:08,523 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5c3c49de4d8e4b148811e3352da5de23 2023-11-02 03:55:08,523 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/5c3c49de4d8e4b148811e3352da5de23, entries=3, sequenceid=24, filesize=5.0 K 2023-11-02 03:55:08,524 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family2/5d2a6c58378a49fcb1a09b66b8f07fc4 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/5d2a6c58378a49fcb1a09b66b8f07fc4 2023-11-02 03:55:08,524 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family2/b2219a8ca28b40ee84d789044077d1a9 2023-11-02 03:55:08,527 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5d2a6c58378a49fcb1a09b66b8f07fc4 2023-11-02 03:55:08,527 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/5d2a6c58378a49fcb1a09b66b8f07fc4, entries=1, sequenceid=24, filesize=5.0 K 2023-11-02 03:55:08,528 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b2219a8ca28b40ee84d789044077d1a9 2023-11-02 03:55:08,528 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for dd77f6ddad975771f984fe41bad1dd90 in 1250ms, sequenceid=24, compaction requested=true 2023-11-02 03:55:08,528 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for dd77f6ddad975771f984fe41bad1dd90: 2023-11-02 03:55:08,528 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:55:08,528 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-02 03:55:08,529 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family1/ea19fcf8ee40499c82c683b52a03a666 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/ea19fcf8ee40499c82c683b52a03a666 2023-11-02 03:55:08,532 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ea19fcf8ee40499c82c683b52a03a666 2023-11-02 03:55:08,532 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/ea19fcf8ee40499c82c683b52a03a666, entries=3, sequenceid=24, filesize=5.0 K 2023-11-02 03:55:08,533 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family2/b2219a8ca28b40ee84d789044077d1a9 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/b2219a8ca28b40ee84d789044077d1a9 2023-11-02 03:55:08,536 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b2219a8ca28b40ee84d789044077d1a9 2023-11-02 03:55:08,536 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/b2219a8ca28b40ee84d789044077d1a9, entries=1, sequenceid=24, filesize=5.0 K 2023-11-02 03:55:08,536 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for b478efa9db26260ecef7476a80a8478c in 1258ms, sequenceid=24, compaction requested=true 2023-11-02 03:55:08,536 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for b478efa9db26260ecef7476a80a8478c: 2023-11-02 03:55:08,536 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:55:08,537 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-02 03:55:08,912 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family2/243a39dc27a84c2b8070975ba83c5c77 2023-11-02 03:55:08,916 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 243a39dc27a84c2b8070975ba83c5c77 2023-11-02 03:55:08,916 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family1/9577dd05d75c4a509caaaf87ca707753 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/9577dd05d75c4a509caaaf87ca707753 2023-11-02 03:55:08,919 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9577dd05d75c4a509caaaf87ca707753 2023-11-02 03:55:08,919 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/9577dd05d75c4a509caaaf87ca707753, entries=3, sequenceid=24, filesize=5.0 K 2023-11-02 03:55:08,920 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family2/243a39dc27a84c2b8070975ba83c5c77 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/243a39dc27a84c2b8070975ba83c5c77 2023-11-02 03:55:08,923 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 243a39dc27a84c2b8070975ba83c5c77 2023-11-02 03:55:08,923 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/243a39dc27a84c2b8070975ba83c5c77, entries=1, sequenceid=24, filesize=5.0 K 2023-11-02 03:55:08,924 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 6ec4530a62d461a9c6949db814d37fd3 in 1646ms, sequenceid=24, compaction requested=true 2023-11-02 03:55:08,924 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 6ec4530a62d461a9c6949db814d37fd3: 2023-11-02 03:55:08,924 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:55:08,924 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-02 03:55:08,924 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-02 03:55:08,924 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:08,924 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-11-02 03:55:08,924 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-11-02 03:55:08,927 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:08,927 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:08,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:08,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:08,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:08,928 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:08,928 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:08,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:08,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:08,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:08,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:08,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:08,929 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-11-02 03:55:08,929 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e5dccfd[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:08,929 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-11-02 03:55:08,929 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:08,931 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:08,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:08,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:08,931 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-11-02 03:55:08,931 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-11-02 03:55:08,931 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:08,931 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:08,931 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:08,933 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-11-02 03:55:08,933 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:08,933 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:08,933 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-11-02 03:55:08,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:08,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:08,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:08,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:08,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:08,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:08,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:08,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:08,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:08,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:08,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:08,935 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-11-02 03:55:08,935 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-11-02 03:55:08,935 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:08,935 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:08,935 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-11-02 03:55:08,935 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:08,937 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:08,937 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:08,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:08,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:08,937 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:08,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:08,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:08,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:08,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:08,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:08,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:08,937 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:08,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:08,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:08,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:08,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:08,939 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:08,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:08,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:08,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-11-02 03:55:08,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:08,941 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:08,941 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:08,942 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:55:08,942 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:08,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:08,942 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-11-02 03:55:08,942 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:08,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:08,942 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-11-02 03:55:08,942 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-11-02 03:55:08,942 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:08,942 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:08,942 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:08,942 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-11-02 03:55:08,942 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-11-02 03:55:08,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:08,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:08,942 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:08,942 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:09,042 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:09,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:09,044 DEBUG [Listener at localhost/43647] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-02 03:55:09,044 DEBUG [Listener at localhost/43647] regionserver.HStore(1912): dd77f6ddad975771f984fe41bad1dd90/family1 is initiating major compaction (all files) 2023-11-02 03:55:09,044 INFO [Listener at localhost/43647] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-02 03:55:09,044 INFO [Listener at localhost/43647] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-02 03:55:09,044 INFO [Listener at localhost/43647] regionserver.HRegion(2259): Starting compaction of dd77f6ddad975771f984fe41bad1dd90/family1 in testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:55:09,044 INFO [Listener at localhost/43647] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/b5e1291b61a04f90918b7f4640c18c61, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/6005881c4ace40b6b1e416da26faa3eb, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/8a84221b334a4e9687250d6733f5fa71, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/161f3f15b77e44a4952ab837e081d9fa, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/5c3c49de4d8e4b148811e3352da5de23] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp, totalSize=26.7 K 2023-11-02 03:55:09,045 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting b5e1291b61a04f90918b7f4640c18c61, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897297923 2023-11-02 03:55:09,045 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 6005881c4ace40b6b1e416da26faa3eb, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897299714 2023-11-02 03:55:09,046 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 8a84221b334a4e9687250d6733f5fa71, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897301515 2023-11-02 03:55:09,046 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 161f3f15b77e44a4952ab837e081d9fa, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897303309 2023-11-02 03:55:09,046 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 5c3c49de4d8e4b148811e3352da5de23, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-02 03:55:09,058 INFO [Listener at localhost/43647] throttle.PressureAwareThroughputController(145): dd77f6ddad975771f984fe41bad1dd90#family1#compaction#107 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:55:09,466 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202311020c1b8786a7f14664b4c89aaba3e166d0_del 2023-11-02 03:55:09,466 INFO [Listener at localhost/43647] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311020c1b8786a7f14664b4c89aaba3e166d0_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311020c1b8786a7f14664b4c89aaba3e166d0_del 2023-11-02 03:55:10,278 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 51f08f9f001f4bf29cabb88fd02be39b 2023-11-02 03:55:10,279 DEBUG [Listener at localhost/43647] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family1/51f08f9f001f4bf29cabb88fd02be39b as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/51f08f9f001f4bf29cabb88fd02be39b 2023-11-02 03:55:10,283 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 51f08f9f001f4bf29cabb88fd02be39b 2023-11-02 03:55:10,284 INFO [Listener at localhost/43647] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in dd77f6ddad975771f984fe41bad1dd90/family1 of dd77f6ddad975771f984fe41bad1dd90 into 51f08f9f001f4bf29cabb88fd02be39b(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-02 03:55:10,284 DEBUG [Listener at localhost/43647] regionserver.HRegion(2289): Compaction status journal for dd77f6ddad975771f984fe41bad1dd90: 2023-11-02 03:55:10,284 DEBUG [Listener at localhost/43647] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-02 03:55:10,284 DEBUG [Listener at localhost/43647] regionserver.HStore(1912): dd77f6ddad975771f984fe41bad1dd90/family2 is initiating major compaction (all files) 2023-11-02 03:55:10,284 INFO [Listener at localhost/43647] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-02 03:55:10,284 INFO [Listener at localhost/43647] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-02 03:55:10,284 INFO [Listener at localhost/43647] regionserver.HRegion(2259): Starting compaction of dd77f6ddad975771f984fe41bad1dd90/family2 in testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:55:10,284 INFO [Listener at localhost/43647] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/26692d5f405449c9bbebf7d285316b79, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/27c8799b53a54db7ab92c9e8b744c603, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/f1a29a576b394c5394ced5cb7815b190, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/0236f03dce144cd980710add026ae931, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/5d2a6c58378a49fcb1a09b66b8f07fc4] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp, totalSize=25.4 K 2023-11-02 03:55:10,285 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 26692d5f405449c9bbebf7d285316b79, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897297923 2023-11-02 03:55:10,285 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 27c8799b53a54db7ab92c9e8b744c603, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897299714 2023-11-02 03:55:10,285 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting f1a29a576b394c5394ced5cb7815b190, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897301515 2023-11-02 03:55:10,286 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 0236f03dce144cd980710add026ae931, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897303309 2023-11-02 03:55:10,286 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 5d2a6c58378a49fcb1a09b66b8f07fc4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-02 03:55:10,296 INFO [Listener at localhost/43647] throttle.PressureAwareThroughputController(145): dd77f6ddad975771f984fe41bad1dd90#family2#compaction#108 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:55:10,307 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/b5e1291b61a04f90918b7f4640c18c61, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/6005881c4ace40b6b1e416da26faa3eb, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/8a84221b334a4e9687250d6733f5fa71, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/161f3f15b77e44a4952ab837e081d9fa, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/5c3c49de4d8e4b148811e3352da5de23] to archive 2023-11-02 03:55:10,307 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:10,309 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/b5e1291b61a04f90918b7f4640c18c61 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/b5e1291b61a04f90918b7f4640c18c61 2023-11-02 03:55:10,310 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/6005881c4ace40b6b1e416da26faa3eb to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/6005881c4ace40b6b1e416da26faa3eb 2023-11-02 03:55:10,311 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/8a84221b334a4e9687250d6733f5fa71 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/8a84221b334a4e9687250d6733f5fa71 2023-11-02 03:55:10,312 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/161f3f15b77e44a4952ab837e081d9fa to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/161f3f15b77e44a4952ab837e081d9fa 2023-11-02 03:55:10,313 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/5c3c49de4d8e4b148811e3352da5de23 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/5c3c49de4d8e4b148811e3352da5de23 2023-11-02 03:55:10,705 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231102be1f3f6929614abe8b2dcfde9a37fa7d_del 2023-11-02 03:55:10,705 INFO [Listener at localhost/43647] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231102be1f3f6929614abe8b2dcfde9a37fa7d_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231102be1f3f6929614abe8b2dcfde9a37fa7d_del 2023-11-02 03:55:11,518 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d60ab40f94ca49499da5cbc2944a941f 2023-11-02 03:55:11,519 DEBUG [Listener at localhost/43647] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family2/d60ab40f94ca49499da5cbc2944a941f as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/d60ab40f94ca49499da5cbc2944a941f 2023-11-02 03:55:11,523 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d60ab40f94ca49499da5cbc2944a941f 2023-11-02 03:55:11,524 INFO [Listener at localhost/43647] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in dd77f6ddad975771f984fe41bad1dd90/family2 of dd77f6ddad975771f984fe41bad1dd90 into d60ab40f94ca49499da5cbc2944a941f(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-02 03:55:11,524 DEBUG [Listener at localhost/43647] regionserver.HRegion(2289): Compaction status journal for dd77f6ddad975771f984fe41bad1dd90: 2023-11-02 03:55:11,524 DEBUG [Listener at localhost/43647] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-02 03:55:11,525 DEBUG [Listener at localhost/43647] regionserver.HStore(1912): 6ec4530a62d461a9c6949db814d37fd3/family1 is initiating major compaction (all files) 2023-11-02 03:55:11,525 INFO [Listener at localhost/43647] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-02 03:55:11,525 INFO [Listener at localhost/43647] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-02 03:55:11,525 INFO [Listener at localhost/43647] regionserver.HRegion(2259): Starting compaction of 6ec4530a62d461a9c6949db814d37fd3/family1 in testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:55:11,525 INFO [Listener at localhost/43647] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/14f9f1322bc6452cadb436583a718478, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/0830c54a581e4ef6bb871baea529ded2, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/a1705e018d254143ae0583e020fae5bc, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/b1ddcc0e1aa7400bb8a5255f06753d0a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/9577dd05d75c4a509caaaf87ca707753] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp, totalSize=26.7 K 2023-11-02 03:55:11,525 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 14f9f1322bc6452cadb436583a718478, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897297922 2023-11-02 03:55:11,526 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 0830c54a581e4ef6bb871baea529ded2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897299713 2023-11-02 03:55:11,526 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting a1705e018d254143ae0583e020fae5bc, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897301515 2023-11-02 03:55:11,527 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting b1ddcc0e1aa7400bb8a5255f06753d0a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897303308 2023-11-02 03:55:11,527 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 9577dd05d75c4a509caaaf87ca707753, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-02 03:55:11,538 INFO [Listener at localhost/43647] throttle.PressureAwareThroughputController(145): 6ec4530a62d461a9c6949db814d37fd3#family1#compaction#109 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:55:11,607 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/26692d5f405449c9bbebf7d285316b79, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/27c8799b53a54db7ab92c9e8b744c603, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/f1a29a576b394c5394ced5cb7815b190, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/0236f03dce144cd980710add026ae931, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/5d2a6c58378a49fcb1a09b66b8f07fc4] to archive 2023-11-02 03:55:11,608 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:11,609 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/26692d5f405449c9bbebf7d285316b79 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/26692d5f405449c9bbebf7d285316b79 2023-11-02 03:55:11,610 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/27c8799b53a54db7ab92c9e8b744c603 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/27c8799b53a54db7ab92c9e8b744c603 2023-11-02 03:55:11,611 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/f1a29a576b394c5394ced5cb7815b190 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/f1a29a576b394c5394ced5cb7815b190 2023-11-02 03:55:11,613 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/0236f03dce144cd980710add026ae931 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/0236f03dce144cd980710add026ae931 2023-11-02 03:55:11,614 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/5d2a6c58378a49fcb1a09b66b8f07fc4 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/5d2a6c58378a49fcb1a09b66b8f07fc4 2023-11-02 03:55:11,945 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023110242ce327a0c2345c08a82f37739a0320e_del 2023-11-02 03:55:11,945 INFO [Listener at localhost/43647] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023110242ce327a0c2345c08a82f37739a0320e_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023110242ce327a0c2345c08a82f37739a0320e_del 2023-11-02 03:55:12,760 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f21ea1acabea4aacb46aca59434b7e63 2023-11-02 03:55:12,760 DEBUG [Listener at localhost/43647] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family1/f21ea1acabea4aacb46aca59434b7e63 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/f21ea1acabea4aacb46aca59434b7e63 2023-11-02 03:55:12,764 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f21ea1acabea4aacb46aca59434b7e63 2023-11-02 03:55:12,765 INFO [Listener at localhost/43647] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6ec4530a62d461a9c6949db814d37fd3/family1 of 6ec4530a62d461a9c6949db814d37fd3 into f21ea1acabea4aacb46aca59434b7e63(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-02 03:55:12,765 DEBUG [Listener at localhost/43647] regionserver.HRegion(2289): Compaction status journal for 6ec4530a62d461a9c6949db814d37fd3: 2023-11-02 03:55:12,765 DEBUG [Listener at localhost/43647] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-02 03:55:12,765 DEBUG [Listener at localhost/43647] regionserver.HStore(1912): 6ec4530a62d461a9c6949db814d37fd3/family2 is initiating major compaction (all files) 2023-11-02 03:55:12,766 INFO [Listener at localhost/43647] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-02 03:55:12,766 INFO [Listener at localhost/43647] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-02 03:55:12,766 INFO [Listener at localhost/43647] regionserver.HRegion(2259): Starting compaction of 6ec4530a62d461a9c6949db814d37fd3/family2 in testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:55:12,766 INFO [Listener at localhost/43647] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/e86f23de8c964b3f8b559674fc99287f, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/c68ad7afbd894381b90008227dceb20a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/1af22c20f93447fb9a8a6c49fdd642ff, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/bf61b264b68a4a5dacd4f9a9803c0106, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/243a39dc27a84c2b8070975ba83c5c77] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp, totalSize=25.4 K 2023-11-02 03:55:12,767 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting e86f23de8c964b3f8b559674fc99287f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897297922 2023-11-02 03:55:12,767 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting c68ad7afbd894381b90008227dceb20a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897299713 2023-11-02 03:55:12,767 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 1af22c20f93447fb9a8a6c49fdd642ff, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897301515 2023-11-02 03:55:12,768 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting bf61b264b68a4a5dacd4f9a9803c0106, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897303308 2023-11-02 03:55:12,768 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 243a39dc27a84c2b8070975ba83c5c77, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-02 03:55:12,780 INFO [Listener at localhost/43647] throttle.PressureAwareThroughputController(145): 6ec4530a62d461a9c6949db814d37fd3#family2#compaction#110 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:55:12,812 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/14f9f1322bc6452cadb436583a718478, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/0830c54a581e4ef6bb871baea529ded2, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/a1705e018d254143ae0583e020fae5bc, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/b1ddcc0e1aa7400bb8a5255f06753d0a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/9577dd05d75c4a509caaaf87ca707753] to archive 2023-11-02 03:55:12,813 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:12,815 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/14f9f1322bc6452cadb436583a718478 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/14f9f1322bc6452cadb436583a718478 2023-11-02 03:55:12,816 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/0830c54a581e4ef6bb871baea529ded2 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/0830c54a581e4ef6bb871baea529ded2 2023-11-02 03:55:12,818 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/a1705e018d254143ae0583e020fae5bc to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/a1705e018d254143ae0583e020fae5bc 2023-11-02 03:55:12,819 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/b1ddcc0e1aa7400bb8a5255f06753d0a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/b1ddcc0e1aa7400bb8a5255f06753d0a 2023-11-02 03:55:12,820 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/9577dd05d75c4a509caaaf87ca707753 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/9577dd05d75c4a509caaaf87ca707753 2023-11-02 03:55:13,194 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231102d0fea6015fa540b18e0f7fa366dfd854_del 2023-11-02 03:55:13,194 INFO [Listener at localhost/43647] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231102d0fea6015fa540b18e0f7fa366dfd854_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231102d0fea6015fa540b18e0f7fa366dfd854_del 2023-11-02 03:55:14,008 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 72c3e075b8244033a3e5d4577a83db92 2023-11-02 03:55:14,009 DEBUG [Listener at localhost/43647] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family2/72c3e075b8244033a3e5d4577a83db92 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/72c3e075b8244033a3e5d4577a83db92 2023-11-02 03:55:14,013 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 72c3e075b8244033a3e5d4577a83db92 2023-11-02 03:55:14,013 INFO [Listener at localhost/43647] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6ec4530a62d461a9c6949db814d37fd3/family2 of 6ec4530a62d461a9c6949db814d37fd3 into 72c3e075b8244033a3e5d4577a83db92(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-02 03:55:14,013 DEBUG [Listener at localhost/43647] regionserver.HRegion(2289): Compaction status journal for 6ec4530a62d461a9c6949db814d37fd3: 2023-11-02 03:55:14,014 DEBUG [Listener at localhost/43647] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-02 03:55:14,014 DEBUG [Listener at localhost/43647] regionserver.HStore(1912): b478efa9db26260ecef7476a80a8478c/family1 is initiating major compaction (all files) 2023-11-02 03:55:14,014 INFO [Listener at localhost/43647] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-02 03:55:14,014 INFO [Listener at localhost/43647] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-02 03:55:14,014 INFO [Listener at localhost/43647] regionserver.HRegion(2259): Starting compaction of b478efa9db26260ecef7476a80a8478c/family1 in testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:55:14,015 INFO [Listener at localhost/43647] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/9ed21d911efc44148a847bf3f3b4cd9b, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/656ab245fd0743afa9d05799e1b639e2, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/203e58c390f74338b4434e5c6a543c55, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/b36e505a69df415fa703d2406810ee79, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/ea19fcf8ee40499c82c683b52a03a666] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp, totalSize=26.7 K 2023-11-02 03:55:14,015 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 9ed21d911efc44148a847bf3f3b4cd9b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897297924 2023-11-02 03:55:14,015 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 656ab245fd0743afa9d05799e1b639e2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897299714 2023-11-02 03:55:14,016 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 203e58c390f74338b4434e5c6a543c55, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897301515 2023-11-02 03:55:14,016 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting b36e505a69df415fa703d2406810ee79, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897303309 2023-11-02 03:55:14,016 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting ea19fcf8ee40499c82c683b52a03a666, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-02 03:55:14,027 INFO [Listener at localhost/43647] throttle.PressureAwareThroughputController(145): b478efa9db26260ecef7476a80a8478c#family1#compaction#111 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:55:14,106 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/e86f23de8c964b3f8b559674fc99287f, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/c68ad7afbd894381b90008227dceb20a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/1af22c20f93447fb9a8a6c49fdd642ff, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/bf61b264b68a4a5dacd4f9a9803c0106, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/243a39dc27a84c2b8070975ba83c5c77] to archive 2023-11-02 03:55:14,107 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:14,108 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/e86f23de8c964b3f8b559674fc99287f to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/e86f23de8c964b3f8b559674fc99287f 2023-11-02 03:55:14,110 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/c68ad7afbd894381b90008227dceb20a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/c68ad7afbd894381b90008227dceb20a 2023-11-02 03:55:14,111 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/1af22c20f93447fb9a8a6c49fdd642ff to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/1af22c20f93447fb9a8a6c49fdd642ff 2023-11-02 03:55:14,111 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/bf61b264b68a4a5dacd4f9a9803c0106 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/bf61b264b68a4a5dacd4f9a9803c0106 2023-11-02 03:55:14,112 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/243a39dc27a84c2b8070975ba83c5c77 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/243a39dc27a84c2b8070975ba83c5c77 2023-11-02 03:55:14,434 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202311028bebc023965847c9aba22f263c722555_del 2023-11-02 03:55:14,434 INFO [Listener at localhost/43647] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311028bebc023965847c9aba22f263c722555_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311028bebc023965847c9aba22f263c722555_del 2023-11-02 03:55:15,249 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 962b8a1414c3401ebac8ac48bdd4c8bd 2023-11-02 03:55:15,250 DEBUG [Listener at localhost/43647] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family1/962b8a1414c3401ebac8ac48bdd4c8bd as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/962b8a1414c3401ebac8ac48bdd4c8bd 2023-11-02 03:55:15,253 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 962b8a1414c3401ebac8ac48bdd4c8bd 2023-11-02 03:55:15,254 INFO [Listener at localhost/43647] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in b478efa9db26260ecef7476a80a8478c/family1 of b478efa9db26260ecef7476a80a8478c into 962b8a1414c3401ebac8ac48bdd4c8bd(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-02 03:55:15,254 DEBUG [Listener at localhost/43647] regionserver.HRegion(2289): Compaction status journal for b478efa9db26260ecef7476a80a8478c: 2023-11-02 03:55:15,254 DEBUG [Listener at localhost/43647] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-02 03:55:15,254 DEBUG [Listener at localhost/43647] regionserver.HStore(1912): b478efa9db26260ecef7476a80a8478c/family2 is initiating major compaction (all files) 2023-11-02 03:55:15,254 INFO [Listener at localhost/43647] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-02 03:55:15,254 INFO [Listener at localhost/43647] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-02 03:55:15,254 INFO [Listener at localhost/43647] regionserver.HRegion(2259): Starting compaction of b478efa9db26260ecef7476a80a8478c/family2 in testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:55:15,255 INFO [Listener at localhost/43647] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/d59a2530e24141fd86350985f46c5be0, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/861b63499b404c75822e982990e9f0c2, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/f850896837b04a078722c50a1c779311, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/cd29ec0b5df94911b3d52331056aa7eb, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/b2219a8ca28b40ee84d789044077d1a9] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp, totalSize=25.4 K 2023-11-02 03:55:15,255 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting d59a2530e24141fd86350985f46c5be0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897297924 2023-11-02 03:55:15,255 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 861b63499b404c75822e982990e9f0c2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897299714 2023-11-02 03:55:15,256 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting f850896837b04a078722c50a1c779311, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897301515 2023-11-02 03:55:15,256 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting cd29ec0b5df94911b3d52331056aa7eb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897303309 2023-11-02 03:55:15,256 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting b2219a8ca28b40ee84d789044077d1a9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-11-02 03:55:15,267 INFO [Listener at localhost/43647] throttle.PressureAwareThroughputController(145): b478efa9db26260ecef7476a80a8478c#family2#compaction#112 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:55:15,307 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/9ed21d911efc44148a847bf3f3b4cd9b, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/656ab245fd0743afa9d05799e1b639e2, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/203e58c390f74338b4434e5c6a543c55, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/b36e505a69df415fa703d2406810ee79, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/ea19fcf8ee40499c82c683b52a03a666] to archive 2023-11-02 03:55:15,308 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:15,310 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/9ed21d911efc44148a847bf3f3b4cd9b to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/9ed21d911efc44148a847bf3f3b4cd9b 2023-11-02 03:55:15,311 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/656ab245fd0743afa9d05799e1b639e2 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/656ab245fd0743afa9d05799e1b639e2 2023-11-02 03:55:15,312 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/203e58c390f74338b4434e5c6a543c55 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/203e58c390f74338b4434e5c6a543c55 2023-11-02 03:55:15,314 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/b36e505a69df415fa703d2406810ee79 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/b36e505a69df415fa703d2406810ee79 2023-11-02 03:55:15,315 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/ea19fcf8ee40499c82c683b52a03a666 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/ea19fcf8ee40499c82c683b52a03a666 2023-11-02 03:55:15,679 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202311023f81b383bb7b45e9a8b8af18f41f1580_del 2023-11-02 03:55:15,679 INFO [Listener at localhost/43647] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311023f81b383bb7b45e9a8b8af18f41f1580_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311023f81b383bb7b45e9a8b8af18f41f1580_del 2023-11-02 03:55:16,492 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3caed0ce1a7a4dbab5ef3adca63a50b2 2023-11-02 03:55:16,493 DEBUG [Listener at localhost/43647] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family2/3caed0ce1a7a4dbab5ef3adca63a50b2 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/3caed0ce1a7a4dbab5ef3adca63a50b2 2023-11-02 03:55:16,497 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3caed0ce1a7a4dbab5ef3adca63a50b2 2023-11-02 03:55:16,498 INFO [Listener at localhost/43647] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in b478efa9db26260ecef7476a80a8478c/family2 of b478efa9db26260ecef7476a80a8478c into 3caed0ce1a7a4dbab5ef3adca63a50b2(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-02 03:55:16,498 DEBUG [Listener at localhost/43647] regionserver.HRegion(2289): Compaction status journal for b478efa9db26260ecef7476a80a8478c: 2023-11-02 03:55:16,504 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-02 03:55:16,508 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-02 03:55:16,509 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/d59a2530e24141fd86350985f46c5be0, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/861b63499b404c75822e982990e9f0c2, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/f850896837b04a078722c50a1c779311, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/cd29ec0b5df94911b3d52331056aa7eb, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/b2219a8ca28b40ee84d789044077d1a9] to archive 2023-11-02 03:55:16,510 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:16,512 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/d59a2530e24141fd86350985f46c5be0 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/d59a2530e24141fd86350985f46c5be0 2023-11-02 03:55:16,512 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-02 03:55:16,513 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-02 03:55:16,514 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/861b63499b404c75822e982990e9f0c2 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/861b63499b404c75822e982990e9f0c2 2023-11-02 03:55:16,521 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/f850896837b04a078722c50a1c779311 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/f850896837b04a078722c50a1c779311 2023-11-02 03:55:16,523 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/cd29ec0b5df94911b3d52331056aa7eb to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/cd29ec0b5df94911b3d52331056aa7eb 2023-11-02 03:55:16,524 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/b2219a8ca28b40ee84d789044077d1a9 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/b2219a8ca28b40ee84d789044077d1a9 2023-11-02 03:55:16,526 INFO [Listener at localhost/43647] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-02 03:55:16,561 INFO [Listener at localhost/43647] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-11-02 03:55:16,564 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023110242ce327a0c2345c08a82f37739a0320e_del 2023-11-02 03:55:16,567 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202311020c1b8786a7f14664b4c89aaba3e166d0_del 2023-11-02 03:55:16,570 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202311028bebc023965847c9aba22f263c722555_del 2023-11-02 03:55:16,570 INFO [Listener at localhost/43647] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-11-02 03:55:16,571 DEBUG [Listener at localhost/43647] zookeeper.ReadOnlyZKClient(139): Connect 0x7a1f1eb7 to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:55:16,576 DEBUG [Listener at localhost/43647] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5aaab782, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:55:16,576 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231102 for table='testCompactionWithHFileLink' and column='family1' 2023-11-02 03:55:16,576 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20231102 for table='testCompactionWithHFileLink' and column='family1' 2023-11-02 03:55:16,576 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231102 for table='testCompactionWithHFileLink' and column='family1' 2023-11-02 03:55:17,041 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311027bfff6b5457b4e079ce62399fc92c665 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311027bfff6b5457b4e079ce62399fc92c665 2023-11-02 03:55:17,042 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231102 table='testCompactionWithHFileLink' column='family1' 2023-11-02 03:55:17,055 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-02 03:55:17,058 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38766, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-02 03:55:17,064 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-02 03:55:17,066 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38840, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-02 03:55:17,073 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231102/family1/bc8b1fc408d241a8925cf3fc55cdf6a6 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-11-02 03:55:17,073 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231102/family1/bc8b1fc408d241a8925cf3fc55cdf6a6}] 2023-11-02 03:55:17,077 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231102/family1/bc8b1fc408d241a8925cf3fc55cdf6a6 for inclusion in 6ec4530a62d461a9c6949db814d37fd3/family1 2023-11-02 03:55:17,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-11-02 03:55:17,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last=1 2023-11-02 03:55:17,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for 6ec4530a62d461a9c6949db814d37fd3: 2023-11-02 03:55:17,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231102/family1/bc8b1fc408d241a8925cf3fc55cdf6a6 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__lett3fq2qp9rbbgjp1sohvt1108c8kg9mjje8b811fvknl9e6lc9r1p4cori04n8/family1/bc8b1fc408d241a8925cf3fc55cdf6a6 2023-11-02 03:55:17,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__lett3fq2qp9rbbgjp1sohvt1108c8kg9mjje8b811fvknl9e6lc9r1p4cori04n8/family1/bc8b1fc408d241a8925cf3fc55cdf6a6 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/b5a9a20749db46faaacd15ca40b650f1_SeqId_29_ 2023-11-02 03:55:17,082 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__lett3fq2qp9rbbgjp1sohvt1108c8kg9mjje8b811fvknl9e6lc9r1p4cori04n8/family1/bc8b1fc408d241a8925cf3fc55cdf6a6 into 6ec4530a62d461a9c6949db814d37fd3/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/b5a9a20749db46faaacd15ca40b650f1_SeqId_29_ - updating store file list. 2023-11-02 03:55:17,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/b5a9a20749db46faaacd15ca40b650f1_SeqId_29_ into 6ec4530a62d461a9c6949db814d37fd3/family1 2023-11-02 03:55:17,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__lett3fq2qp9rbbgjp1sohvt1108c8kg9mjje8b811fvknl9e6lc9r1p4cori04n8/family1/bc8b1fc408d241a8925cf3fc55cdf6a6 into 6ec4530a62d461a9c6949db814d37fd3/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/b5a9a20749db46faaacd15ca40b650f1_SeqId_29_) 2023-11-02 03:55:17,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__lett3fq2qp9rbbgjp1sohvt1108c8kg9mjje8b811fvknl9e6lc9r1p4cori04n8/family1/bc8b1fc408d241a8925cf3fc55cdf6a6 2023-11-02 03:55:17,086 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231102 table='testCompactionWithHFileLink' column='family1' 2023-11-02 03:55:17,087 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:17,088 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311020e34f72f161545fe81dd972e46735ff4 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311020e34f72f161545fe81dd972e46735ff4 2023-11-02 03:55:17,088 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311021353ebc9c5414069887b922ea4aaf9c0 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311021353ebc9c5414069887b922ea4aaf9c0 2023-11-02 03:55:17,089 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311022554ea023fd24ffb84021913dc7f96c0 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311022554ea023fd24ffb84021913dc7f96c0 2023-11-02 03:55:17,090 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231102e561bf0308514c75b8062bddf947eb1c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231102e561bf0308514c75b8062bddf947eb1c 2023-11-02 03:55:17,090 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-02 03:55:17,090 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231102' 2023-11-02 03:55:17,090 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231102 for table='testCompactionWithHFileLink' and column='family1' 2023-11-02 03:55:17,442 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231102ad0bcf3e079a43a8acaa5ef5975d2b8e to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231102ad0bcf3e079a43a8acaa5ef5975d2b8e 2023-11-02 03:55:17,442 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311028030df47c0db489c9ae99cffd0c0dd20 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311028030df47c0db489c9ae99cffd0c0dd20 2023-11-02 03:55:17,443 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231102 table='testCompactionWithHFileLink' column='family1' 2023-11-02 03:55:17,443 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231102 table='testCompactionWithHFileLink' column='family1' 2023-11-02 03:55:17,467 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231102/family1/748dd0dce923491d876e43ebdc1b8df1 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-11-02 03:55:17,467 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231102/family1/cfd338f2eb224f468f3f0c305bb58df7 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-11-02 03:55:17,468 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231102/family1/748dd0dce923491d876e43ebdc1b8df1}] 2023-11-02 03:55:17,468 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231102/family1/cfd338f2eb224f468f3f0c305bb58df7}] 2023-11-02 03:55:17,470 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231102/family1/748dd0dce923491d876e43ebdc1b8df1 for inclusion in b478efa9db26260ecef7476a80a8478c/family1 2023-11-02 03:55:17,470 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231102/family1/cfd338f2eb224f468f3f0c305bb58df7 for inclusion in dd77f6ddad975771f984fe41bad1dd90/family1 2023-11-02 03:55:17,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-11-02 03:55:17,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-11-02 03:55:17,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(777): Region bounds: first=2 last= 2023-11-02 03:55:17,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first=1 last=2 2023-11-02 03:55:17,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for b478efa9db26260ecef7476a80a8478c: 2023-11-02 03:55:17,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for dd77f6ddad975771f984fe41bad1dd90: 2023-11-02 03:55:17,475 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231102/family1/748dd0dce923491d876e43ebdc1b8df1 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__vvlqe3di327l7tn34mknt1t67qs651ju4m3oi4rkkkum2e3rpb020nj419tj40mv/family1/748dd0dce923491d876e43ebdc1b8df1 2023-11-02 03:55:17,475 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231102/family1/cfd338f2eb224f468f3f0c305bb58df7 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__9fdehuvj4me258s2dtbnbtfqgdd1h0vukugmt7cb41am6lc29lnsm9h1onvlf0rd/family1/cfd338f2eb224f468f3f0c305bb58df7 2023-11-02 03:55:17,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__vvlqe3di327l7tn34mknt1t67qs651ju4m3oi4rkkkum2e3rpb020nj419tj40mv/family1/748dd0dce923491d876e43ebdc1b8df1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/ef0c9436251a4dccbe39ea4860e455c6_SeqId_29_ 2023-11-02 03:55:17,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__9fdehuvj4me258s2dtbnbtfqgdd1h0vukugmt7cb41am6lc29lnsm9h1onvlf0rd/family1/cfd338f2eb224f468f3f0c305bb58df7 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/4c36b133fd8a4b4880269de6175269ba_SeqId_29_ 2023-11-02 03:55:17,477 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__vvlqe3di327l7tn34mknt1t67qs651ju4m3oi4rkkkum2e3rpb020nj419tj40mv/family1/748dd0dce923491d876e43ebdc1b8df1 into b478efa9db26260ecef7476a80a8478c/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/ef0c9436251a4dccbe39ea4860e455c6_SeqId_29_ - updating store file list. 2023-11-02 03:55:17,477 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__9fdehuvj4me258s2dtbnbtfqgdd1h0vukugmt7cb41am6lc29lnsm9h1onvlf0rd/family1/cfd338f2eb224f468f3f0c305bb58df7 into dd77f6ddad975771f984fe41bad1dd90/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/4c36b133fd8a4b4880269de6175269ba_SeqId_29_ - updating store file list. 2023-11-02 03:55:17,480 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/ef0c9436251a4dccbe39ea4860e455c6_SeqId_29_ into b478efa9db26260ecef7476a80a8478c/family1 2023-11-02 03:55:17,480 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__vvlqe3di327l7tn34mknt1t67qs651ju4m3oi4rkkkum2e3rpb020nj419tj40mv/family1/748dd0dce923491d876e43ebdc1b8df1 into b478efa9db26260ecef7476a80a8478c/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/ef0c9436251a4dccbe39ea4860e455c6_SeqId_29_) 2023-11-02 03:55:17,480 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/4c36b133fd8a4b4880269de6175269ba_SeqId_29_ into dd77f6ddad975771f984fe41bad1dd90/family1 2023-11-02 03:55:17,480 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__9fdehuvj4me258s2dtbnbtfqgdd1h0vukugmt7cb41am6lc29lnsm9h1onvlf0rd/family1/cfd338f2eb224f468f3f0c305bb58df7 into dd77f6ddad975771f984fe41bad1dd90/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/4c36b133fd8a4b4880269de6175269ba_SeqId_29_) 2023-11-02 03:55:17,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__vvlqe3di327l7tn34mknt1t67qs651ju4m3oi4rkkkum2e3rpb020nj419tj40mv/family1/748dd0dce923491d876e43ebdc1b8df1 2023-11-02 03:55:17,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__9fdehuvj4me258s2dtbnbtfqgdd1h0vukugmt7cb41am6lc29lnsm9h1onvlf0rd/family1/cfd338f2eb224f468f3f0c305bb58df7 2023-11-02 03:55:17,482 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231102 table='testCompactionWithHFileLink' column='family1' 2023-11-02 03:55:17,482 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231102 table='testCompactionWithHFileLink' column='family1' 2023-11-02 03:55:17,482 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:17,482 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:17,484 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311020b8bff03978d4222a97f41a288302e66 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311020b8bff03978d4222a97f41a288302e66 2023-11-02 03:55:17,484 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311021cc42c22257a40eb9c5aeb2784b3ad66 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311021cc42c22257a40eb9c5aeb2784b3ad66 2023-11-02 03:55:17,485 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023110210ea62b1fd9c46d38b55f31359a0d585 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023110210ea62b1fd9c46d38b55f31359a0d585 2023-11-02 03:55:17,485 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231102342f97ac7df94b6d87dfc0a98523bbb9 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231102342f97ac7df94b6d87dfc0a98523bbb9 2023-11-02 03:55:17,486 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311022eecb97c58c94d5c88d39499e2c15dd6 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311022eecb97c58c94d5c88d39499e2c15dd6 2023-11-02 03:55:17,486 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311024ec670ce8f05437389ed33e14c71a7da to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311024ec670ce8f05437389ed33e14c71a7da 2023-11-02 03:55:17,488 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311026c16c5c49f724abdae7b6b32320f65ff to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311026c16c5c49f724abdae7b6b32320f65ff 2023-11-02 03:55:17,488 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-02 03:55:17,488 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311023beda44052714bfba055af33c67ecdf6 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311023beda44052714bfba055af33c67ecdf6 2023-11-02 03:55:17,488 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231102' 2023-11-02 03:55:17,488 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-02 03:55:17,488 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231102' 2023-11-02 03:55:17,488 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20231102 for table='testCompactionWithHFileLink' and column='family1' 2023-11-02 03:55:17,488 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231102 for table='testCompactionWithHFileLink' and column='family1' 2023-11-02 03:55:17,488 INFO [Listener at localhost/43647] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-11-02 03:55:17,489 INFO [Listener at localhost/43647] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-11-02 03:55:17,489 INFO [Listener at localhost/43647] compactions.PartitionedMobCompactor(367): [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023110242ce327a0c2345c08a82f37739a0320e_del] 2023-11-02 03:55:17,490 DEBUG [Listener at localhost/43647] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:17,491 DEBUG [Listener at localhost/43647] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023110242ce327a0c2345c08a82f37739a0320e_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023110242ce327a0c2345c08a82f37739a0320e_del 2023-11-02 03:55:17,491 INFO [Listener at localhost/43647] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-11-02 03:55:17,492 INFO [Listener at localhost/43647] compactions.PartitionedMobCompactor(367): [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311020c1b8786a7f14664b4c89aaba3e166d0_del] 2023-11-02 03:55:17,492 DEBUG [Listener at localhost/43647] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:17,493 DEBUG [Listener at localhost/43647] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311020c1b8786a7f14664b4c89aaba3e166d0_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202311020c1b8786a7f14664b4c89aaba3e166d0_del 2023-11-02 03:55:17,493 INFO [Listener at localhost/43647] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-11-02 03:55:17,493 INFO [Listener at localhost/43647] compactions.PartitionedMobCompactor(367): [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311028bebc023965847c9aba22f263c722555_del] 2023-11-02 03:55:17,494 DEBUG [Listener at localhost/43647] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:17,495 DEBUG [Listener at localhost/43647] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311028bebc023965847c9aba22f263c722555_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311028bebc023965847c9aba22f263c722555_del 2023-11-02 03:55:17,495 INFO [Listener at localhost/43647] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-11-02 03:55:17,498 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-02 03:55:17,500 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-02 03:55:17,503 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-02 03:55:17,504 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-02 03:55:17,509 INFO [Listener at localhost/43647] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-11-02 03:55:17,509 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-11-02 03:55:17,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-11-02 03:55:17,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-11-02 03:55:17,512 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897317512"}]},"ts":"1698897317512"} 2023-11-02 03:55:17,513 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-11-02 03:55:17,515 INFO [PEWorker-4] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-11-02 03:55:17,516 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6ec4530a62d461a9c6949db814d37fd3, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd77f6ddad975771f984fe41bad1dd90, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b478efa9db26260ecef7476a80a8478c, UNASSIGN}] 2023-11-02 03:55:17,517 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6ec4530a62d461a9c6949db814d37fd3, UNASSIGN 2023-11-02 03:55:17,517 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd77f6ddad975771f984fe41bad1dd90, UNASSIGN 2023-11-02 03:55:17,517 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b478efa9db26260ecef7476a80a8478c, UNASSIGN 2023-11-02 03:55:17,517 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=dd77f6ddad975771f984fe41bad1dd90, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:17,517 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=b478efa9db26260ecef7476a80a8478c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:17,517 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=6ec4530a62d461a9c6949db814d37fd3, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:17,517 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698897317517"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897317517"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897317517"}]},"ts":"1698897317517"} 2023-11-02 03:55:17,518 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698897317517"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897317517"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897317517"}]},"ts":"1698897317517"} 2023-11-02 03:55:17,517 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698897317517"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897317517"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897317517"}]},"ts":"1698897317517"} 2023-11-02 03:55:17,519 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; CloseRegionProcedure dd77f6ddad975771f984fe41bad1dd90, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:55:17,519 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=93, ppid=89, state=RUNNABLE; CloseRegionProcedure 6ec4530a62d461a9c6949db814d37fd3, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:55:17,520 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=94, ppid=91, state=RUNNABLE; CloseRegionProcedure b478efa9db26260ecef7476a80a8478c, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:55:17,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-11-02 03:55:17,670 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:17,671 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close b478efa9db26260ecef7476a80a8478c 2023-11-02 03:55:17,671 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b478efa9db26260ecef7476a80a8478c, disabling compactions & flushes 2023-11-02 03:55:17,671 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:55:17,671 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:55:17,671 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. after waiting 0 ms 2023-11-02 03:55:17,671 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:55:17,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-11-02 03:55:17,678 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:55:17,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b478efa9db26260ecef7476a80a8478c: 2023-11-02 03:55:17,680 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed b478efa9db26260ecef7476a80a8478c 2023-11-02 03:55:17,680 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:55:17,680 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6ec4530a62d461a9c6949db814d37fd3, disabling compactions & flushes 2023-11-02 03:55:17,680 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:55:17,680 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:55:17,680 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. after waiting 0 ms 2023-11-02 03:55:17,680 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:55:17,680 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=b478efa9db26260ecef7476a80a8478c, regionState=CLOSED 2023-11-02 03:55:17,680 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698897317680"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897317680"}]},"ts":"1698897317680"} 2023-11-02 03:55:17,684 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=94, resume processing ppid=91 2023-11-02 03:55:17,684 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=94, ppid=91, state=SUCCESS; CloseRegionProcedure b478efa9db26260ecef7476a80a8478c, server=jenkins-hbase4.apache.org,36627,1698897201574 in 162 msec 2023-11-02 03:55:17,685 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-11-02 03:55:17,685 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b478efa9db26260ecef7476a80a8478c, UNASSIGN in 168 msec 2023-11-02 03:55:17,686 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:55:17,686 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6ec4530a62d461a9c6949db814d37fd3: 2023-11-02 03:55:17,687 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:55:17,687 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:55:17,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing dd77f6ddad975771f984fe41bad1dd90, disabling compactions & flushes 2023-11-02 03:55:17,687 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:55:17,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:55:17,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. after waiting 0 ms 2023-11-02 03:55:17,688 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:55:17,688 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=6ec4530a62d461a9c6949db814d37fd3, regionState=CLOSED 2023-11-02 03:55:17,688 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698897317688"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897317688"}]},"ts":"1698897317688"} 2023-11-02 03:55:17,691 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=93, resume processing ppid=89 2023-11-02 03:55:17,691 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=93, ppid=89, state=SUCCESS; CloseRegionProcedure 6ec4530a62d461a9c6949db814d37fd3, server=jenkins-hbase4.apache.org,36627,1698897201574 in 171 msec 2023-11-02 03:55:17,692 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6ec4530a62d461a9c6949db814d37fd3, UNASSIGN in 175 msec 2023-11-02 03:55:17,692 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-11-02 03:55:17,693 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:55:17,693 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for dd77f6ddad975771f984fe41bad1dd90: 2023-11-02 03:55:17,694 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:55:17,694 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=dd77f6ddad975771f984fe41bad1dd90, regionState=CLOSED 2023-11-02 03:55:17,694 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698897317694"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897317694"}]},"ts":"1698897317694"} 2023-11-02 03:55:17,696 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=92, resume processing ppid=90 2023-11-02 03:55:17,696 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=92, ppid=90, state=SUCCESS; CloseRegionProcedure dd77f6ddad975771f984fe41bad1dd90, server=jenkins-hbase4.apache.org,36627,1698897201574 in 176 msec 2023-11-02 03:55:17,697 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=90, resume processing ppid=88 2023-11-02 03:55:17,697 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd77f6ddad975771f984fe41bad1dd90, UNASSIGN in 180 msec 2023-11-02 03:55:17,698 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897317698"}]},"ts":"1698897317698"} 2023-11-02 03:55:17,699 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-11-02 03:55:17,700 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-11-02 03:55:17,702 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 192 msec 2023-11-02 03:55:17,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-11-02 03:55:17,814 INFO [Listener at localhost/43647] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-11-02 03:55:17,824 INFO [Listener at localhost/43647] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1698897305088-1698897317824 2023-11-02 03:55:17,825 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1583): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1698897305088-1698897317824 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-02 03:55:17,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1698897317825 (current time:1698897317825). 2023-11-02 03:55:17,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-11-02 03:55:17,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] zookeeper.ReadOnlyZKClient(139): Connect 0x7476533f to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:55:17,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5d0d6143, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:55:17,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-02 03:55:17,833 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38776, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-02 03:55:17,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7476533f to 127.0.0.1:65185 2023-11-02 03:55:17,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:55:17,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] snapshot.SnapshotManager(637): No existing snapshot, attempting snapshot... 2023-11-02 03:55:17,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] snapshot.SnapshotManager(695): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1698897305088-1698897317824 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-02 03:55:17,837 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] snapshot.SnapshotManager(519): Unable to retrieve ACL status for path: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.hbase-snapshot, current working dir path: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1698897305088-1698897317824 org.apache.hadoop.hdfs.protocol.AclException: The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false. at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.checkAclsConfigFlag(FSDirAclOp.java:197) at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.getAclStatus(FSDirAclOp.java:150) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:6744) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getAclStatus(NameNodeRpcServer.java:2065) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getAclStatus(ClientNamenodeProtocolServerSideTranslatorPB.java:1378) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.getAclStatus(DFSClient.java:2736) at org.apache.hadoop.hdfs.DistributedFileSystem$51.doCall(DistributedFileSystem.java:2294) at org.apache.hadoop.hdfs.DistributedFileSystem$51.doCall(DistributedFileSystem.java:2291) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getAclStatus(DistributedFileSystem.java:2291) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.updateWorkingDirAclsIfRequired(SnapshotManager.java:517) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.prepareToTakeSnapshot(SnapshotManager.java:490) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.snapshotDisabledTable(SnapshotManager.java:541) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshotInternal(SnapshotManager.java:698) at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.takeSnapshot(SnapshotManager.java:623) at org.apache.hadoop.hbase.master.MasterRpcServices.snapshot(MasterRpcServices.java:1588) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.AclException): The ACL operation has been rejected. Support for ACLs has been disabled by setting dfs.namenode.acls.enabled to false. at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.checkAclsConfigFlag(FSDirAclOp.java:197) at org.apache.hadoop.hdfs.server.namenode.FSDirAclOp.getAclStatus(FSDirAclOp.java:150) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAclStatus(FSNamesystem.java:6744) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getAclStatus(NameNodeRpcServer.java:2065) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getAclStatus(ClientNamenodeProtocolServerSideTranslatorPB.java:1378) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.getAclStatus(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getAclStatus(ClientNamenodeProtocolTranslatorPB.java:1385) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.getAclStatus(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getAclStatus(DFSClient.java:2734) ... 15 more 2023-11-02 03:55:17,839 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-02 03:55:17,839 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-02 03:55:17,840 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1698897917840 2023-11-02 03:55:17,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] snapshot.SnapshotManager(700): Started snapshot: { ss=hbase-failsafe-snaptb-1698897305088-1698897317824 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-02 03:55:17,840 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1698897305088-1698897317824 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-11-02 03:55:17,840 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1698897305088-1698897317824 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:17,840 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-11-02 03:55:17,940 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-02 03:55:17,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1698897305088-1698897317824 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-02 03:55:17,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] snapshot.SnapshotManager(406): Snapshoting '{ ss=hbase-failsafe-snaptb-1698897305088-1698897317824 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-02 03:55:17,941 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-11-02 03:55:18,142 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-02 03:55:18,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1698897305088-1698897317824 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-02 03:55:18,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] snapshot.SnapshotManager(406): Snapshoting '{ ss=hbase-failsafe-snaptb-1698897305088-1698897317824 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-02 03:55:18,143 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-11-02 03:55:18,247 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1698897305088-1698897317824 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-11-02 03:55:18,249 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-11-02 03:55:18,249 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-11-02 03:55:18,250 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-11-02 03:55:18,250 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-11-02 03:55:18,250 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-11-02 03:55:18,250 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-11-02 03:55:18,250 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-11-02 03:55:18,250 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-11-02 03:55:18,253 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/b5a9a20749db46faaacd15ca40b650f1_SeqId_29_, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/f21ea1acabea4aacb46aca59434b7e63] hfiles 2023-11-02 03:55:18,253 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/962b8a1414c3401ebac8ac48bdd4c8bd, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/ef0c9436251a4dccbe39ea4860e455c6_SeqId_29_] hfiles 2023-11-02 03:55:18,253 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/b5a9a20749db46faaacd15ca40b650f1_SeqId_29_ 2023-11-02 03:55:18,253 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/4c36b133fd8a4b4880269de6175269ba_SeqId_29_, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/51f08f9f001f4bf29cabb88fd02be39b] hfiles 2023-11-02 03:55:18,253 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/962b8a1414c3401ebac8ac48bdd4c8bd 2023-11-02 03:55:18,253 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/4c36b133fd8a4b4880269de6175269ba_SeqId_29_ 2023-11-02 03:55:18,253 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/ef0c9436251a4dccbe39ea4860e455c6_SeqId_29_ 2023-11-02 03:55:18,253 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/f21ea1acabea4aacb46aca59434b7e63 2023-11-02 03:55:18,253 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/51f08f9f001f4bf29cabb88fd02be39b 2023-11-02 03:55:18,254 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231102ad0bcf3e079a43a8acaa5ef5975d2b8e, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311028030df47c0db489c9ae99cffd0c0dd20, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311027bfff6b5457b4e079ce62399fc92c665] hfiles 2023-11-02 03:55:18,254 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231102ad0bcf3e079a43a8acaa5ef5975d2b8e 2023-11-02 03:55:18,254 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311028030df47c0db489c9ae99cffd0c0dd20 2023-11-02 03:55:18,254 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311027bfff6b5457b4e079ce62399fc92c665 2023-11-02 03:55:18,254 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/3caed0ce1a7a4dbab5ef3adca63a50b2] hfiles 2023-11-02 03:55:18,254 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/3caed0ce1a7a4dbab5ef3adca63a50b2 2023-11-02 03:55:18,254 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/72c3e075b8244033a3e5d4577a83db92] hfiles 2023-11-02 03:55:18,255 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/d60ab40f94ca49499da5cbc2944a941f] hfiles 2023-11-02 03:55:18,255 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/72c3e075b8244033a3e5d4577a83db92 2023-11-02 03:55:18,255 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/d60ab40f94ca49499da5cbc2944a941f 2023-11-02 03:55:18,265 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023110218ec4ff8aa4243d4bf9a3153acdf7dff, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023110231ae017da0d549489f6bcf43217b8a22, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311026b3cc1b9116d44cb91aa244c6aeb83ca, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231102be1f3f6929614abe8b2dcfde9a37fa7d_del, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231102c641bcec015b4c3da7f4e1d9141d5cb7, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311023f81b383bb7b45e9a8b8af18f41f1580_del, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231102541edba285f246e29a687e91402c074b, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311025d35ed5ec5934294baaab5f677a57e8a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023110264d619ba95e0487f9c5d1cb23cabc3c8, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231102ddb80c75b59640e3bb5dc44794ee82ea, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023110299c01bb84a054f5c919cf9eaba97a698, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231102cee98bd818574201b98024dea9e0da98, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231102d0fea6015fa540b18e0f7fa366dfd854_del, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231102d63c3f81e2214c1587fd5ec7bcdd427e, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231102e801ed4a3d4948cf8c154e0dc2f14b74] hfiles 2023-11-02 03:55:18,265 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023110218ec4ff8aa4243d4bf9a3153acdf7dff 2023-11-02 03:55:18,265 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023110231ae017da0d549489f6bcf43217b8a22 2023-11-02 03:55:18,265 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202311026b3cc1b9116d44cb91aa244c6aeb83ca 2023-11-02 03:55:18,265 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231102be1f3f6929614abe8b2dcfde9a37fa7d_del 2023-11-02 03:55:18,265 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231102c641bcec015b4c3da7f4e1d9141d5cb7 2023-11-02 03:55:18,265 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311023f81b383bb7b45e9a8b8af18f41f1580_del 2023-11-02 03:55:18,265 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231102541edba285f246e29a687e91402c074b 2023-11-02 03:55:18,265 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311025d35ed5ec5934294baaab5f677a57e8a 2023-11-02 03:55:18,265 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023110264d619ba95e0487f9c5d1cb23cabc3c8 2023-11-02 03:55:18,265 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20231102ddb80c75b59640e3bb5dc44794ee82ea 2023-11-02 03:55:18,265 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023110299c01bb84a054f5c919cf9eaba97a698 2023-11-02 03:55:18,265 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231102cee98bd818574201b98024dea9e0da98 2023-11-02 03:55:18,265 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231102d0fea6015fa540b18e0f7fa366dfd854_del 2023-11-02 03:55:18,265 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231102d63c3f81e2214c1587fd5ec7bcdd427e 2023-11-02 03:55:18,265 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231102e801ed4a3d4948cf8c154e0dc2f14b74 2023-11-02 03:55:18,443 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-02 03:55:18,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1698897305088-1698897317824 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-02 03:55:18,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] snapshot.SnapshotManager(406): Snapshoting '{ ss=hbase-failsafe-snaptb-1698897305088-1698897317824 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-11-02 03:55:18,444 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-11-02 03:55:18,671 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1698897305088-1698897317824 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-11-02 03:55:18,671 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1698897305088-1698897317824 2023-11-02 03:55:18,671 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1698897305088-1698897317824 2023-11-02 03:55:18,705 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1698897305088-1698897317824 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.hbase-snapshot/hbase-failsafe-snaptb-1698897305088-1698897317824 2023-11-02 03:55:18,706 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1698897305088-1698897317824 of table testCompactionWithHFileLink completed 2023-11-02 03:55:18,706 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1698897305088-1698897317824 2023-11-02 03:55:18,706 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1698897305088-1698897317824 2023-11-02 03:55:18,706 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1698897305088-1698897317824 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1698897317840Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1698897305088-1698897317824 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1698897318247 (+407 ms)Consolidate snapshot: hbase-failsafe-snaptb-1698897305088-1698897317824 at 1698897318671 (+424 ms)Loading Region manifests for hbase-failsafe-snaptb-1698897305088-1698897317824 at 1698897318671Writing data manifest for hbase-failsafe-snaptb-1698897305088-1698897317824 at 1698897318674 (+3 ms)Verifying snapshot: hbase-failsafe-snaptb-1698897305088-1698897317824 at 1698897318686 (+12 ms)Snapshot hbase-failsafe-snaptb-1698897305088-1698897317824 of table testCompactionWithHFileLink completed at 1698897318706 (+20 ms) 2023-11-02 03:55:18,708 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-11-02 03:55:18,709 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 870 msec 2023-11-02 03:55:18,944 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-11-02 03:55:18,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1698897305088-1698897317824 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-11-02 03:55:18,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] snapshot.SnapshotManager(403): Snapshot '{ ss=hbase-failsafe-snaptb-1698897305088-1698897317824 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-11-02 03:55:18,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1698897305088" table: "testCompactionWithHFileLink" creation_time: 1698897305098 type: FLUSH version: 2 ttl: 0 ) 2023-11-02 03:55:18,985 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] snapshot.SnapshotManager(916): Restore snapshot=snaptb-1698897305088 as table=testCompactionWithHFileLink 2023-11-02 03:55:18,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-11-02 03:55:19,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-11-02 03:55:19,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-11-02 03:55:19,395 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-11-02 03:55:19,396 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-11-02 03:55:19,396 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-11-02 03:55:19,396 INFO [PEWorker-3] procedure.RestoreSnapshotProcedure(384): Starting restore snapshot={ ss=snaptb-1698897305088 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-11-02 03:55:19,401 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1698897305088" table: "testCompactionWithHFileLink" creation_time: 1698897305098 type: FLUSH version: 2 ttl: 0 2023-11-02 03:55:19,402 DEBUG [PEWorker-3] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink 2023-11-02 03:55:19,407 DEBUG [PEWorker-3] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-11-02 03:55:19,410 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231102ad0bcf3e079a43a8acaa5ef5975d2b8e to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231102ad0bcf3e079a43a8acaa5ef5975d2b8e 2023-11-02 03:55:19,411 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311027bfff6b5457b4e079ce62399fc92c665 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202311027bfff6b5457b4e079ce62399fc92c665 2023-11-02 03:55:19,412 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311028030df47c0db489c9ae99cffd0c0dd20 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202311028030df47c0db489c9ae99cffd0c0dd20 2023-11-02 03:55:19,412 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202311021cc42c22257a40eb9c5aeb2784b3ad66 of snapshot=snaptb-1698897305088 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-02 03:55:19,415 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20231102342f97ac7df94b6d87dfc0a98523bbb9 of snapshot=snaptb-1698897305088 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-02 03:55:19,418 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202311024ec670ce8f05437389ed33e14c71a7da of snapshot=snaptb-1698897305088 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-02 03:55:19,420 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202311026c16c5c49f724abdae7b6b32320f65ff of snapshot=snaptb-1698897305088 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-02 03:55:19,423 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202311020b8bff03978d4222a97f41a288302e66 of snapshot=snaptb-1698897305088 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-02 03:55:19,426 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023110210ea62b1fd9c46d38b55f31359a0d585 of snapshot=snaptb-1698897305088 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-02 03:55:19,429 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202311022eecb97c58c94d5c88d39499e2c15dd6 of snapshot=snaptb-1698897305088 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-02 03:55:19,432 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202311023beda44052714bfba055af33c67ecdf6 of snapshot=snaptb-1698897305088 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-02 03:55:19,434 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202311020e34f72f161545fe81dd972e46735ff4 of snapshot=snaptb-1698897305088 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-02 03:55:19,437 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202311021353ebc9c5414069887b922ea4aaf9c0 of snapshot=snaptb-1698897305088 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-02 03:55:19,440 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202311022554ea023fd24ffb84021913dc7f96c0 of snapshot=snaptb-1698897305088 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-02 03:55:19,443 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20231102e561bf0308514c75b8062bddf947eb1c of snapshot=snaptb-1698897305088 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-11-02 03:55:19,447 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231102be1f3f6929614abe8b2dcfde9a37fa7d_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20231102be1f3f6929614abe8b2dcfde9a37fa7d_del 2023-11-02 03:55:19,449 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311023f81b383bb7b45e9a8b8af18f41f1580_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202311023f81b383bb7b45e9a8b8af18f41f1580_del 2023-11-02 03:55:19,450 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231102d0fea6015fa540b18e0f7fa366dfd854_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20231102d0fea6015fa540b18e0f7fa366dfd854_del 2023-11-02 03:55:19,450 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: 6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:55:19,450 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: b478efa9db26260ecef7476a80a8478c 2023-11-02 03:55:19,450 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:55:19,453 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/b5a9a20749db46faaacd15ca40b650f1_SeqId_29_ to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/b5a9a20749db46faaacd15ca40b650f1_SeqId_29_ 2023-11-02 03:55:19,453 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/962b8a1414c3401ebac8ac48bdd4c8bd to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/962b8a1414c3401ebac8ac48bdd4c8bd 2023-11-02 03:55:19,453 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/51f08f9f001f4bf29cabb88fd02be39b to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/51f08f9f001f4bf29cabb88fd02be39b 2023-11-02 03:55:19,455 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/f21ea1acabea4aacb46aca59434b7e63 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/f21ea1acabea4aacb46aca59434b7e63 2023-11-02 03:55:19,455 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/ef0c9436251a4dccbe39ea4860e455c6_SeqId_29_ to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/ef0c9436251a4dccbe39ea4860e455c6_SeqId_29_ 2023-11-02 03:55:19,455 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 9ed21d911efc44148a847bf3f3b4cd9b of snapshot=snaptb-1698897305088 to region=b478efa9db26260ecef7476a80a8478c table=testCompactionWithHFileLink 2023-11-02 03:55:19,455 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 14f9f1322bc6452cadb436583a718478 of snapshot=snaptb-1698897305088 to region=6ec4530a62d461a9c6949db814d37fd3 table=testCompactionWithHFileLink 2023-11-02 03:55:19,455 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/4c36b133fd8a4b4880269de6175269ba_SeqId_29_ to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/4c36b133fd8a4b4880269de6175269ba_SeqId_29_ 2023-11-02 03:55:19,455 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b5e1291b61a04f90918b7f4640c18c61 of snapshot=snaptb-1698897305088 to region=dd77f6ddad975771f984fe41bad1dd90 table=testCompactionWithHFileLink 2023-11-02 03:55:19,459 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 656ab245fd0743afa9d05799e1b639e2 of snapshot=snaptb-1698897305088 to region=b478efa9db26260ecef7476a80a8478c table=testCompactionWithHFileLink 2023-11-02 03:55:19,460 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 6005881c4ace40b6b1e416da26faa3eb of snapshot=snaptb-1698897305088 to region=dd77f6ddad975771f984fe41bad1dd90 table=testCompactionWithHFileLink 2023-11-02 03:55:19,461 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0830c54a581e4ef6bb871baea529ded2 of snapshot=snaptb-1698897305088 to region=6ec4530a62d461a9c6949db814d37fd3 table=testCompactionWithHFileLink 2023-11-02 03:55:19,467 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 203e58c390f74338b4434e5c6a543c55 of snapshot=snaptb-1698897305088 to region=b478efa9db26260ecef7476a80a8478c table=testCompactionWithHFileLink 2023-11-02 03:55:19,467 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a1705e018d254143ae0583e020fae5bc of snapshot=snaptb-1698897305088 to region=6ec4530a62d461a9c6949db814d37fd3 table=testCompactionWithHFileLink 2023-11-02 03:55:19,467 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8a84221b334a4e9687250d6733f5fa71 of snapshot=snaptb-1698897305088 to region=dd77f6ddad975771f984fe41bad1dd90 table=testCompactionWithHFileLink 2023-11-02 03:55:19,471 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b36e505a69df415fa703d2406810ee79 of snapshot=snaptb-1698897305088 to region=b478efa9db26260ecef7476a80a8478c table=testCompactionWithHFileLink 2023-11-02 03:55:19,471 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink b1ddcc0e1aa7400bb8a5255f06753d0a of snapshot=snaptb-1698897305088 to region=6ec4530a62d461a9c6949db814d37fd3 table=testCompactionWithHFileLink 2023-11-02 03:55:19,471 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 161f3f15b77e44a4952ab837e081d9fa of snapshot=snaptb-1698897305088 to region=dd77f6ddad975771f984fe41bad1dd90 table=testCompactionWithHFileLink 2023-11-02 03:55:19,476 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/3caed0ce1a7a4dbab5ef3adca63a50b2 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/3caed0ce1a7a4dbab5ef3adca63a50b2 2023-11-02 03:55:19,476 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d59a2530e24141fd86350985f46c5be0 of snapshot=snaptb-1698897305088 to region=b478efa9db26260ecef7476a80a8478c table=testCompactionWithHFileLink 2023-11-02 03:55:19,476 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/72c3e075b8244033a3e5d4577a83db92 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/72c3e075b8244033a3e5d4577a83db92 2023-11-02 03:55:19,476 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e86f23de8c964b3f8b559674fc99287f of snapshot=snaptb-1698897305088 to region=6ec4530a62d461a9c6949db814d37fd3 table=testCompactionWithHFileLink 2023-11-02 03:55:19,476 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/d60ab40f94ca49499da5cbc2944a941f to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/d60ab40f94ca49499da5cbc2944a941f 2023-11-02 03:55:19,476 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 26692d5f405449c9bbebf7d285316b79 of snapshot=snaptb-1698897305088 to region=dd77f6ddad975771f984fe41bad1dd90 table=testCompactionWithHFileLink 2023-11-02 03:55:19,487 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c68ad7afbd894381b90008227dceb20a of snapshot=snaptb-1698897305088 to region=6ec4530a62d461a9c6949db814d37fd3 table=testCompactionWithHFileLink 2023-11-02 03:55:19,488 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 861b63499b404c75822e982990e9f0c2 of snapshot=snaptb-1698897305088 to region=b478efa9db26260ecef7476a80a8478c table=testCompactionWithHFileLink 2023-11-02 03:55:19,488 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 27c8799b53a54db7ab92c9e8b744c603 of snapshot=snaptb-1698897305088 to region=dd77f6ddad975771f984fe41bad1dd90 table=testCompactionWithHFileLink 2023-11-02 03:55:19,492 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 1af22c20f93447fb9a8a6c49fdd642ff of snapshot=snaptb-1698897305088 to region=6ec4530a62d461a9c6949db814d37fd3 table=testCompactionWithHFileLink 2023-11-02 03:55:19,494 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f1a29a576b394c5394ced5cb7815b190 of snapshot=snaptb-1698897305088 to region=dd77f6ddad975771f984fe41bad1dd90 table=testCompactionWithHFileLink 2023-11-02 03:55:19,501 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f850896837b04a078722c50a1c779311 of snapshot=snaptb-1698897305088 to region=b478efa9db26260ecef7476a80a8478c table=testCompactionWithHFileLink 2023-11-02 03:55:19,501 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink bf61b264b68a4a5dacd4f9a9803c0106 of snapshot=snaptb-1698897305088 to region=6ec4530a62d461a9c6949db814d37fd3 table=testCompactionWithHFileLink 2023-11-02 03:55:19,502 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0236f03dce144cd980710add026ae931 of snapshot=snaptb-1698897305088 to region=dd77f6ddad975771f984fe41bad1dd90 table=testCompactionWithHFileLink 2023-11-02 03:55:19,504 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink cd29ec0b5df94911b3d52331056aa7eb of snapshot=snaptb-1698897305088 to region=b478efa9db26260ecef7476a80a8478c table=testCompactionWithHFileLink 2023-11-02 03:55:19,507 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1698897305088" table: "testCompactionWithHFileLink" creation_time: 1698897305098 type: FLUSH version: 2 ttl: 0 2023-11-02 03:55:19,508 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1698897319508"}]},"ts":"9223372036854775807"} 2023-11-02 03:55:19,508 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1698897319508"}]},"ts":"9223372036854775807"} 2023-11-02 03:55:19,508 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1698897319508"}]},"ts":"9223372036854775807"} 2023-11-02 03:55:19,510 INFO [PEWorker-3] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-11-02 03:55:19,510 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 6ec4530a62d461a9c6949db814d37fd3, NAME => 'testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => b478efa9db26260ecef7476a80a8478c, NAME => 'testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => dd77f6ddad975771f984fe41bad1dd90, NAME => 'testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90.', STARTKEY => '1', ENDKEY => '2'}] 2023-11-02 03:55:19,510 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698897319509"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897319509"}]},"ts":"1698897319509"} 2023-11-02 03:55:19,510 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698897319509"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897319509"}]},"ts":"1698897319509"} 2023-11-02 03:55:19,510 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698897319509"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897319509"}]},"ts":"1698897319509"} 2023-11-02 03:55:19,512 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-11-02 03:55:19,512 INFO [PEWorker-3] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-11-02 03:55:19,512 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 6ec4530a62d461a9c6949db814d37fd3, NAME => 'testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => b478efa9db26260ecef7476a80a8478c, NAME => 'testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => dd77f6ddad975771f984fe41bad1dd90, NAME => 'testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90.', STARTKEY => '1', ENDKEY => '2'}] 2023-11-02 03:55:19,512 INFO [PEWorker-3] procedure.RestoreSnapshotProcedure(457): Restore snapshot={ ss=snaptb-1698897305088 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-11-02 03:55:19,514 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1698897305088" table: "testCompactionWithHFileLink" creation_time: 1698897305098 type: FLUSH version: 2 ttl: 0 ) in 536 msec 2023-11-02 03:55:19,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-11-02 03:55:19,591 INFO [Listener at localhost/43647] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-11-02 03:55:19,591 INFO [Listener at localhost/43647] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1698897305088-1698897317824 2023-11-02 03:55:19,595 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(751): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1698897305088-1698897317824" 2023-11-02 03:55:19,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] snapshot.SnapshotManager(335): Deleting snapshot: hbase-failsafe-snaptb-1698897305088-1698897317824 2023-11-02 03:55:19,599 INFO [Listener at localhost/43647] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-11-02 03:55:19,603 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.HMaster$10(2385): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-11-02 03:55:19,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-11-02 03:55:19,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-11-02 03:55:19,612 INFO [PEWorker-1] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-11-02 03:55:19,612 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897319612"}]},"ts":"1698897319612"} 2023-11-02 03:55:19,613 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-11-02 03:55:19,615 DEBUG [PEWorker-1] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-11-02 03:55:19,615 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6ec4530a62d461a9c6949db814d37fd3, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd77f6ddad975771f984fe41bad1dd90, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b478efa9db26260ecef7476a80a8478c, ASSIGN}] 2023-11-02 03:55:19,617 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd77f6ddad975771f984fe41bad1dd90, ASSIGN 2023-11-02 03:55:19,617 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b478efa9db26260ecef7476a80a8478c, ASSIGN 2023-11-02 03:55:19,617 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6ec4530a62d461a9c6949db814d37fd3, ASSIGN 2023-11-02 03:55:19,617 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd77f6ddad975771f984fe41bad1dd90, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-11-02 03:55:19,617 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b478efa9db26260ecef7476a80a8478c, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-11-02 03:55:19,617 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6ec4530a62d461a9c6949db814d37fd3, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-11-02 03:55:19,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-11-02 03:55:19,769 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=6ec4530a62d461a9c6949db814d37fd3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:19,769 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=b478efa9db26260ecef7476a80a8478c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:19,769 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=dd77f6ddad975771f984fe41bad1dd90, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:19,769 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698897319769"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897319769"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897319769"}]},"ts":"1698897319769"} 2023-11-02 03:55:19,769 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698897319769"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897319769"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897319769"}]},"ts":"1698897319769"} 2023-11-02 03:55:19,769 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698897319769"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897319769"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897319769"}]},"ts":"1698897319769"} 2023-11-02 03:55:19,770 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=101, ppid=98, state=RUNNABLE; OpenRegionProcedure 6ec4530a62d461a9c6949db814d37fd3, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:55:19,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=102, ppid=99, state=RUNNABLE; OpenRegionProcedure dd77f6ddad975771f984fe41bad1dd90, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:55:19,772 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=103, ppid=100, state=RUNNABLE; OpenRegionProcedure b478efa9db26260ecef7476a80a8478c, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:55:19,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-11-02 03:55:19,922 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:19,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:55:19,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => dd77f6ddad975771f984fe41bad1dd90, NAME => 'testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90.', STARTKEY => '1', ENDKEY => '2'} 2023-11-02 03:55:19,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:55:19,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:55:19,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:55:19,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:55:19,926 INFO [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:55:19,927 DEBUG [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1 2023-11-02 03:55:19,927 DEBUG [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1 2023-11-02 03:55:19,927 INFO [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dd77f6ddad975771f984fe41bad1dd90 columnFamilyName family1 2023-11-02 03:55:19,937 DEBUG [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-161f3f15b77e44a4952ab837e081d9fa 2023-11-02 03:55:19,942 DEBUG [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-6005881c4ace40b6b1e416da26faa3eb 2023-11-02 03:55:19,948 DEBUG [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-8a84221b334a4e9687250d6733f5fa71 2023-11-02 03:55:19,953 DEBUG [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-b5e1291b61a04f90918b7f4640c18c61 2023-11-02 03:55:19,953 INFO [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] regionserver.HStore(310): Store=dd77f6ddad975771f984fe41bad1dd90/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:55:19,954 INFO [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:55:19,955 DEBUG [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2 2023-11-02 03:55:19,955 DEBUG [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2 2023-11-02 03:55:19,955 INFO [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dd77f6ddad975771f984fe41bad1dd90 columnFamilyName family2 2023-11-02 03:55:19,962 DEBUG [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-0236f03dce144cd980710add026ae931 2023-11-02 03:55:19,967 DEBUG [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-26692d5f405449c9bbebf7d285316b79 2023-11-02 03:55:19,972 DEBUG [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-27c8799b53a54db7ab92c9e8b744c603 2023-11-02 03:55:19,978 DEBUG [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-f1a29a576b394c5394ced5cb7815b190 2023-11-02 03:55:19,978 INFO [StoreOpener-dd77f6ddad975771f984fe41bad1dd90-1] regionserver.HStore(310): Store=dd77f6ddad975771f984fe41bad1dd90/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:55:19,979 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:55:19,980 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:55:19,981 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-02 03:55:19,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:55:19,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened dd77f6ddad975771f984fe41bad1dd90; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71478967, jitterRate=0.06511960923671722}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-02 03:55:19,983 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for dd77f6ddad975771f984fe41bad1dd90: 2023-11-02 03:55:19,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90., pid=102, masterSystemTime=1698897319922 2023-11-02 03:55:19,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:55:19,985 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-02 03:55:19,985 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:55:19,985 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-02 03:55:19,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:55:19,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:55:19,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:55:19,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6ec4530a62d461a9c6949db814d37fd3, NAME => 'testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3.', STARTKEY => '', ENDKEY => '1'} 2023-11-02 03:55:19,987 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=dd77f6ddad975771f984fe41bad1dd90, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:19,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:55:19,987 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698897319987"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897319987"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897319987"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897319987"}]},"ts":"1698897319987"} 2023-11-02 03:55:19,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:55:19,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:55:19,987 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:55:19,988 INFO [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:55:19,989 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-11-02 03:55:19,989 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.HStore(1912): dd77f6ddad975771f984fe41bad1dd90/family2 is initiating minor compaction (all files) 2023-11-02 03:55:19,989 INFO [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.HRegion(2259): Starting compaction of dd77f6ddad975771f984fe41bad1dd90/family2 in testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:55:19,989 INFO [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-26692d5f405449c9bbebf7d285316b79, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-27c8799b53a54db7ab92c9e8b744c603, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-f1a29a576b394c5394ced5cb7815b190, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-0236f03dce144cd980710add026ae931] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp, totalSize=20.4 K 2023-11-02 03:55:19,989 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-11-02 03:55:19,989 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1912): dd77f6ddad975771f984fe41bad1dd90/family1 is initiating minor compaction (all files) 2023-11-02 03:55:19,989 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of dd77f6ddad975771f984fe41bad1dd90/family1 in testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:55:19,989 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-b5e1291b61a04f90918b7f4640c18c61, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-6005881c4ace40b6b1e416da26faa3eb, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-8a84221b334a4e9687250d6733f5fa71, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-161f3f15b77e44a4952ab837e081d9fa] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp, totalSize=21.6 K 2023-11-02 03:55:19,989 DEBUG [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1 2023-11-02 03:55:19,989 DEBUG [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1 2023-11-02 03:55:19,990 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-26692d5f405449c9bbebf7d285316b79, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897297923 2023-11-02 03:55:19,990 INFO [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ec4530a62d461a9c6949db814d37fd3 columnFamilyName family1 2023-11-02 03:55:19,990 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-b5e1291b61a04f90918b7f4640c18c61, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897297923 2023-11-02 03:55:19,990 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-27c8799b53a54db7ab92c9e8b744c603, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897299714 2023-11-02 03:55:19,991 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=102, resume processing ppid=99 2023-11-02 03:55:19,991 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=102, ppid=99, state=SUCCESS; OpenRegionProcedure dd77f6ddad975771f984fe41bad1dd90, server=jenkins-hbase4.apache.org,36627,1698897201574 in 218 msec 2023-11-02 03:55:19,991 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-6005881c4ace40b6b1e416da26faa3eb, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897299714 2023-11-02 03:55:19,991 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-f1a29a576b394c5394ced5cb7815b190, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897301515 2023-11-02 03:55:19,992 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=dd77f6ddad975771f984fe41bad1dd90, ASSIGN in 376 msec 2023-11-02 03:55:19,992 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-8a84221b334a4e9687250d6733f5fa71, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897301515 2023-11-02 03:55:19,992 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-0236f03dce144cd980710add026ae931, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897303309 2023-11-02 03:55:19,992 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-161f3f15b77e44a4952ab837e081d9fa, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897303309 2023-11-02 03:55:20,001 DEBUG [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-0830c54a581e4ef6bb871baea529ded2 2023-11-02 03:55:20,014 DEBUG [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-14f9f1322bc6452cadb436583a718478 2023-11-02 03:55:20,022 DEBUG [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-a1705e018d254143ae0583e020fae5bc 2023-11-02 03:55:20,029 INFO [RS:0;jenkins-hbase4:36627-longCompactions-0] throttle.PressureAwareThroughputController(145): dd77f6ddad975771f984fe41bad1dd90#family2#compaction#113 average throughput is 0.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 53.85 MB/second 2023-11-02 03:55:20,033 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] throttle.PressureAwareThroughputController(145): dd77f6ddad975771f984fe41bad1dd90#family1#compaction#114 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-02 03:55:20,033 DEBUG [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-b1ddcc0e1aa7400bb8a5255f06753d0a 2023-11-02 03:55:20,037 INFO [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] regionserver.HStore(310): Store=6ec4530a62d461a9c6949db814d37fd3/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:55:20,037 INFO [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:55:20,038 DEBUG [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2 2023-11-02 03:55:20,038 DEBUG [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2 2023-11-02 03:55:20,039 INFO [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ec4530a62d461a9c6949db814d37fd3 columnFamilyName family2 2023-11-02 03:55:20,045 DEBUG [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-1af22c20f93447fb9a8a6c49fdd642ff 2023-11-02 03:55:20,049 DEBUG [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-bf61b264b68a4a5dacd4f9a9803c0106 2023-11-02 03:55:20,052 DEBUG [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-c68ad7afbd894381b90008227dceb20a 2023-11-02 03:55:20,056 DEBUG [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-e86f23de8c964b3f8b559674fc99287f 2023-11-02 03:55:20,056 INFO [StoreOpener-6ec4530a62d461a9c6949db814d37fd3-1] regionserver.HStore(310): Store=6ec4530a62d461a9c6949db814d37fd3/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:55:20,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:55:20,058 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:55:20,059 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-02 03:55:20,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:55:20,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6ec4530a62d461a9c6949db814d37fd3; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75224140, jitterRate=0.12092703580856323}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-02 03:55:20,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6ec4530a62d461a9c6949db814d37fd3: 2023-11-02 03:55:20,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3., pid=101, masterSystemTime=1698897319922 2023-11-02 03:55:20,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-02 03:55:20,061 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-11-02 03:55:20,062 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:55:20,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:55:20,062 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:55:20,063 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b478efa9db26260ecef7476a80a8478c, NAME => 'testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c.', STARTKEY => '2', ENDKEY => ''} 2023-11-02 03:55:20,063 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=6ec4530a62d461a9c6949db814d37fd3, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:20,063 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink b478efa9db26260ecef7476a80a8478c 2023-11-02 03:55:20,063 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:55:20,063 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b478efa9db26260ecef7476a80a8478c 2023-11-02 03:55:20,063 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b478efa9db26260ecef7476a80a8478c 2023-11-02 03:55:20,063 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698897320063"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897320063"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897320063"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897320063"}]},"ts":"1698897320063"} 2023-11-02 03:55:20,064 INFO [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b478efa9db26260ecef7476a80a8478c 2023-11-02 03:55:20,065 DEBUG [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1 2023-11-02 03:55:20,065 DEBUG [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1 2023-11-02 03:55:20,065 INFO [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b478efa9db26260ecef7476a80a8478c columnFamilyName family1 2023-11-02 03:55:20,066 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=101, resume processing ppid=98 2023-11-02 03:55:20,066 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=101, ppid=98, state=SUCCESS; OpenRegionProcedure 6ec4530a62d461a9c6949db814d37fd3, server=jenkins-hbase4.apache.org,36627,1698897201574 in 294 msec 2023-11-02 03:55:20,066 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=6ec4530a62d461a9c6949db814d37fd3, ASSIGN in 451 msec 2023-11-02 03:55:20,073 DEBUG [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-203e58c390f74338b4434e5c6a543c55 2023-11-02 03:55:20,077 DEBUG [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-656ab245fd0743afa9d05799e1b639e2 2023-11-02 03:55:20,081 DEBUG [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-9ed21d911efc44148a847bf3f3b4cd9b 2023-11-02 03:55:20,086 DEBUG [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-b36e505a69df415fa703d2406810ee79 2023-11-02 03:55:20,086 INFO [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] regionserver.HStore(310): Store=b478efa9db26260ecef7476a80a8478c/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:55:20,087 INFO [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region b478efa9db26260ecef7476a80a8478c 2023-11-02 03:55:20,088 DEBUG [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2 2023-11-02 03:55:20,088 DEBUG [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2 2023-11-02 03:55:20,088 INFO [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b478efa9db26260ecef7476a80a8478c columnFamilyName family2 2023-11-02 03:55:20,097 DEBUG [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-861b63499b404c75822e982990e9f0c2 2023-11-02 03:55:20,104 DEBUG [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-cd29ec0b5df94911b3d52331056aa7eb 2023-11-02 03:55:20,112 DEBUG [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-d59a2530e24141fd86350985f46c5be0 2023-11-02 03:55:20,119 DEBUG [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] regionserver.HStore(539): loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-f850896837b04a078722c50a1c779311 2023-11-02 03:55:20,119 INFO [StoreOpener-b478efa9db26260ecef7476a80a8478c-1] regionserver.HStore(310): Store=b478efa9db26260ecef7476a80a8478c/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:55:20,120 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c 2023-11-02 03:55:20,121 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c 2023-11-02 03:55:20,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-02 03:55:20,124 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b478efa9db26260ecef7476a80a8478c 2023-11-02 03:55:20,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b478efa9db26260ecef7476a80a8478c; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70438013, jitterRate=0.04960818588733673}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-02 03:55:20,125 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b478efa9db26260ecef7476a80a8478c: 2023-11-02 03:55:20,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c., pid=103, masterSystemTime=1698897319922 2023-11-02 03:55:20,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-11-02 03:55:20,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-11-02 03:55:20,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:55:20,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:55:20,128 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=b478efa9db26260ecef7476a80a8478c, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:20,128 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1698897320128"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897320128"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897320128"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897320128"}]},"ts":"1698897320128"} 2023-11-02 03:55:20,131 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=103, resume processing ppid=100 2023-11-02 03:55:20,131 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=103, ppid=100, state=SUCCESS; OpenRegionProcedure b478efa9db26260ecef7476a80a8478c, server=jenkins-hbase4.apache.org,36627,1698897201574 in 357 msec 2023-11-02 03:55:20,133 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=100, resume processing ppid=97 2023-11-02 03:55:20,133 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=b478efa9db26260ecef7476a80a8478c, ASSIGN in 516 msec 2023-11-02 03:55:20,134 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897320133"}]},"ts":"1698897320133"} 2023-11-02 03:55:20,135 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-11-02 03:55:20,136 INFO [PEWorker-4] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-11-02 03:55:20,138 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 532 msec 2023-11-02 03:55:20,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=97 2023-11-02 03:55:20,215 INFO [Listener at localhost/43647] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-11-02 03:55:20,227 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-02 03:55:20,234 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-02 03:55:20,243 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-02 03:55:20,244 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-02 03:55:20,250 INFO [Listener at localhost/43647] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-02 03:55:20,263 INFO [Listener at localhost/43647] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-11-02 03:55:20,264 INFO [Listener at localhost/43647] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-11-02 03:55:20,264 DEBUG [Listener at localhost/43647] zookeeper.ReadOnlyZKClient(139): Connect 0x345a3dae to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:55:20,269 DEBUG [Listener at localhost/43647] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@565de4d9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:55:20,269 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231102 for table='testCompactionWithHFileLink' and column='family1' 2023-11-02 03:55:20,269 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231102 for table='testCompactionWithHFileLink' and column='family1' 2023-11-02 03:55:20,269 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20231102 for table='testCompactionWithHFileLink' and column='family1' 2023-11-02 03:55:20,755 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231102f396c8d259cb4ec3bd3b9648ac34c9ce to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20231102f396c8d259cb4ec3bd3b9648ac34c9ce 2023-11-02 03:55:20,756 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231102 table='testCompactionWithHFileLink' column='family1' 2023-11-02 03:55:20,769 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-02 03:55:20,772 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38790, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-02 03:55:20,779 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-02 03:55:20,782 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38850, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-02 03:55:20,794 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231102/family1/e7ec3e13f27843dca1e856c34aa13799 first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-11-02 03:55:20,794 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=32 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231102/family1/e7ec3e13f27843dca1e856c34aa13799}] 2023-11-02 03:55:20,799 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231102/family1/e7ec3e13f27843dca1e856c34aa13799 for inclusion in 6ec4530a62d461a9c6949db814d37fd3/family1 2023-11-02 03:55:20,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-11-02 03:55:20,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last=1 2023-11-02 03:55:20,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for 6ec4530a62d461a9c6949db814d37fd3: 2023-11-02 03:55:20,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231102/family1/e7ec3e13f27843dca1e856c34aa13799 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__2078eo7mgd6vpne4r7bqecm70m8jl5bv1iilvqdsvkvsjcpd10i5su6qltde1uhe/family1/e7ec3e13f27843dca1e856c34aa13799 2023-11-02 03:55:20,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__2078eo7mgd6vpne4r7bqecm70m8jl5bv1iilvqdsvkvsjcpd10i5su6qltde1uhe/family1/e7ec3e13f27843dca1e856c34aa13799 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/dbc04f93e9da408fbd7fc483661f964d_SeqId_34_ 2023-11-02 03:55:20,804 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__2078eo7mgd6vpne4r7bqecm70m8jl5bv1iilvqdsvkvsjcpd10i5su6qltde1uhe/family1/e7ec3e13f27843dca1e856c34aa13799 into 6ec4530a62d461a9c6949db814d37fd3/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/dbc04f93e9da408fbd7fc483661f964d_SeqId_34_ - updating store file list. 2023-11-02 03:55:20,808 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/dbc04f93e9da408fbd7fc483661f964d_SeqId_34_ into 6ec4530a62d461a9c6949db814d37fd3/family1 2023-11-02 03:55:20,808 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__2078eo7mgd6vpne4r7bqecm70m8jl5bv1iilvqdsvkvsjcpd10i5su6qltde1uhe/family1/e7ec3e13f27843dca1e856c34aa13799 into 6ec4530a62d461a9c6949db814d37fd3/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/dbc04f93e9da408fbd7fc483661f964d_SeqId_34_) 2023-11-02 03:55:20,809 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__2078eo7mgd6vpne4r7bqecm70m8jl5bv1iilvqdsvkvsjcpd10i5su6qltde1uhe/family1/e7ec3e13f27843dca1e856c34aa13799 2023-11-02 03:55:20,809 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-02 03:55:20,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-11-02 03:55:20,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(1912): 6ec4530a62d461a9c6949db814d37fd3/family1 is initiating minor compaction (all files) 2023-11-02 03:55:20,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@59505167; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-11-02 03:55:20,813 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231102 table='testCompactionWithHFileLink' column='family1' 2023-11-02 03:55:20,814 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:20,815 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202311020e34f72f161545fe81dd972e46735ff4 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202311020e34f72f161545fe81dd972e46735ff4 2023-11-02 03:55:20,816 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202311021353ebc9c5414069887b922ea4aaf9c0 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202311021353ebc9c5414069887b922ea4aaf9c0 2023-11-02 03:55:20,817 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202311022554ea023fd24ffb84021913dc7f96c0 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202311022554ea023fd24ffb84021913dc7f96c0 2023-11-02 03:55:20,818 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20231102e561bf0308514c75b8062bddf947eb1c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20231102e561bf0308514c75b8062bddf947eb1c 2023-11-02 03:55:20,818 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-02 03:55:20,818 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20231102' 2023-11-02 03:55:20,818 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231102 for table='testCompactionWithHFileLink' and column='family1' 2023-11-02 03:55:20,860 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family1/d8f3e9bd88854ec4a33336d9fb438315 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/d8f3e9bd88854ec4a33336d9fb438315 2023-11-02 03:55:20,862 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/.tmp/family2/5cee9701c3f6427d9f9d6068fbbc4748 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/5cee9701c3f6427d9f9d6068fbbc4748 2023-11-02 03:55:20,866 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in dd77f6ddad975771f984fe41bad1dd90/family1 of dd77f6ddad975771f984fe41bad1dd90 into d8f3e9bd88854ec4a33336d9fb438315(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-02 03:55:20,866 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for dd77f6ddad975771f984fe41bad1dd90: 2023-11-02 03:55:20,866 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90., storeName=dd77f6ddad975771f984fe41bad1dd90/family1, priority=12, startTime=1698897319984; duration=0sec 2023-11-02 03:55:20,866 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-11-02 03:55:20,866 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 6ec4530a62d461a9c6949db814d37fd3/family1 in testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:55:20,866 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-14f9f1322bc6452cadb436583a718478, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-0830c54a581e4ef6bb871baea529ded2, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-a1705e018d254143ae0583e020fae5bc, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-b1ddcc0e1aa7400bb8a5255f06753d0a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/dbc04f93e9da408fbd7fc483661f964d_SeqId_34_] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp, totalSize=28.9 K 2023-11-02 03:55:20,866 INFO [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in dd77f6ddad975771f984fe41bad1dd90/family2 of dd77f6ddad975771f984fe41bad1dd90 into 5cee9701c3f6427d9f9d6068fbbc4748(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-02 03:55:20,867 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for dd77f6ddad975771f984fe41bad1dd90: 2023-11-02 03:55:20,867 INFO [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90., storeName=dd77f6ddad975771f984fe41bad1dd90/family2, priority=12, startTime=1698897319985; duration=0sec 2023-11-02 03:55:20,867 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-11-02 03:55:20,867 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-02 03:55:20,867 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-14f9f1322bc6452cadb436583a718478, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897297922 2023-11-02 03:55:20,868 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-0830c54a581e4ef6bb871baea529ded2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897299713 2023-11-02 03:55:20,869 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-a1705e018d254143ae0583e020fae5bc, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897301515 2023-11-02 03:55:20,869 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-b1ddcc0e1aa7400bb8a5255f06753d0a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897303308 2023-11-02 03:55:20,869 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-11-02 03:55:20,870 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.HStore(1912): 6ec4530a62d461a9c6949db814d37fd3/family2 is initiating minor compaction (all files) 2023-11-02 03:55:20,870 INFO [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 6ec4530a62d461a9c6949db814d37fd3/family2 in testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:55:20,870 INFO [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-e86f23de8c964b3f8b559674fc99287f, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-c68ad7afbd894381b90008227dceb20a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-1af22c20f93447fb9a8a6c49fdd642ff, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-bf61b264b68a4a5dacd4f9a9803c0106] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp, totalSize=20.4 K 2023-11-02 03:55:20,870 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting dbc04f93e9da408fbd7fc483661f964d_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1698897297922 2023-11-02 03:55:20,870 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-e86f23de8c964b3f8b559674fc99287f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897297922 2023-11-02 03:55:20,871 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-c68ad7afbd894381b90008227dceb20a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897299713 2023-11-02 03:55:20,872 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-1af22c20f93447fb9a8a6c49fdd642ff, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897301515 2023-11-02 03:55:20,872 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-bf61b264b68a4a5dacd4f9a9803c0106, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897303308 2023-11-02 03:55:20,887 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6ec4530a62d461a9c6949db814d37fd3#family1#compaction#115 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-02 03:55:20,897 INFO [RS:0;jenkins-hbase4:36627-longCompactions-0] throttle.PressureAwareThroughputController(145): 6ec4530a62d461a9c6949db814d37fd3#family2#compaction#116 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-02 03:55:20,908 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-b5e1291b61a04f90918b7f4640c18c61, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-6005881c4ace40b6b1e416da26faa3eb, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-8a84221b334a4e9687250d6733f5fa71, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-161f3f15b77e44a4952ab837e081d9fa] to archive 2023-11-02 03:55:20,908 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:20,910 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-b5e1291b61a04f90918b7f4640c18c61 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-b5e1291b61a04f90918b7f4640c18c61 2023-11-02 03:55:20,911 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-6005881c4ace40b6b1e416da26faa3eb to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-6005881c4ace40b6b1e416da26faa3eb 2023-11-02 03:55:20,913 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-8a84221b334a4e9687250d6733f5fa71 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-8a84221b334a4e9687250d6733f5fa71 2023-11-02 03:55:20,914 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-161f3f15b77e44a4952ab837e081d9fa to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-161f3f15b77e44a4952ab837e081d9fa 2023-11-02 03:55:20,920 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-26692d5f405449c9bbebf7d285316b79, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-27c8799b53a54db7ab92c9e8b744c603, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-f1a29a576b394c5394ced5cb7815b190, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-0236f03dce144cd980710add026ae931] to archive 2023-11-02 03:55:20,921 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:20,922 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-26692d5f405449c9bbebf7d285316b79 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-26692d5f405449c9bbebf7d285316b79 2023-11-02 03:55:20,923 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-27c8799b53a54db7ab92c9e8b744c603 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-27c8799b53a54db7ab92c9e8b744c603 2023-11-02 03:55:20,925 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-f1a29a576b394c5394ced5cb7815b190 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-f1a29a576b394c5394ced5cb7815b190 2023-11-02 03:55:20,926 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-0236f03dce144cd980710add026ae931 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family2/testCompactionWithHFileLink=dd77f6ddad975771f984fe41bad1dd90-0236f03dce144cd980710add026ae931 2023-11-02 03:55:21,154 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023110236f86274fc7b45c89013712d03a7d5ba to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023110236f86274fc7b45c89013712d03a7d5ba 2023-11-02 03:55:21,155 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231102d9e59ce0bf324bc1a777bf41a8c6b419 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20231102d9e59ce0bf324bc1a777bf41a8c6b419 2023-11-02 03:55:21,155 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231102 table='testCompactionWithHFileLink' column='family1' 2023-11-02 03:55:21,156 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231102 table='testCompactionWithHFileLink' column='family1' 2023-11-02 03:55:21,182 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231102/family1/eeb7343e1ed24c9da9ce96eedc45ec87 first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-11-02 03:55:21,182 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231102/family1/eeb7343e1ed24c9da9ce96eedc45ec87}] 2023-11-02 03:55:21,182 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231102/family1/c457d135938f4c3c9730d2de0f3bffac first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-11-02 03:55:21,183 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231102/family1/c457d135938f4c3c9730d2de0f3bffac}] 2023-11-02 03:55:21,185 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231102/family1/c457d135938f4c3c9730d2de0f3bffac for inclusion in b478efa9db26260ecef7476a80a8478c/family1 2023-11-02 03:55:21,185 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231102/family1/eeb7343e1ed24c9da9ce96eedc45ec87 for inclusion in dd77f6ddad975771f984fe41bad1dd90/family1 2023-11-02 03:55:21,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-11-02 03:55:21,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(777): Region bounds: first=2 last= 2023-11-02 03:55:21,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for b478efa9db26260ecef7476a80a8478c: 2023-11-02 03:55:21,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-11-02 03:55:21,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first=1 last=2 2023-11-02 03:55:21,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for dd77f6ddad975771f984fe41bad1dd90: 2023-11-02 03:55:21,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231102/family1/c457d135938f4c3c9730d2de0f3bffac to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__hm12l0gmu3iaidvl76nv94qp0eh1mli9tnsbdhn439l1ao85grer2qntusgf9cq9/family1/c457d135938f4c3c9730d2de0f3bffac 2023-11-02 03:55:21,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231102/family1/eeb7343e1ed24c9da9ce96eedc45ec87 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__ba356dqp47ubbk1gn1b8oqgm66loqhtol4np7445rp4i3kmli9qp4pfo928kg2tg/family1/eeb7343e1ed24c9da9ce96eedc45ec87 2023-11-02 03:55:21,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__hm12l0gmu3iaidvl76nv94qp0eh1mli9tnsbdhn439l1ao85grer2qntusgf9cq9/family1/c457d135938f4c3c9730d2de0f3bffac as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/5072a50899a94f73992578744f854b43_SeqId_34_ 2023-11-02 03:55:21,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__ba356dqp47ubbk1gn1b8oqgm66loqhtol4np7445rp4i3kmli9qp4pfo928kg2tg/family1/eeb7343e1ed24c9da9ce96eedc45ec87 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/aa3c8cd0c897447bb3b7b8258069da59_SeqId_36_ 2023-11-02 03:55:21,191 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__hm12l0gmu3iaidvl76nv94qp0eh1mli9tnsbdhn439l1ao85grer2qntusgf9cq9/family1/c457d135938f4c3c9730d2de0f3bffac into b478efa9db26260ecef7476a80a8478c/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/5072a50899a94f73992578744f854b43_SeqId_34_ - updating store file list. 2023-11-02 03:55:21,192 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__ba356dqp47ubbk1gn1b8oqgm66loqhtol4np7445rp4i3kmli9qp4pfo928kg2tg/family1/eeb7343e1ed24c9da9ce96eedc45ec87 into dd77f6ddad975771f984fe41bad1dd90/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/aa3c8cd0c897447bb3b7b8258069da59_SeqId_36_ - updating store file list. 2023-11-02 03:55:21,195 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/5072a50899a94f73992578744f854b43_SeqId_34_ into b478efa9db26260ecef7476a80a8478c/family1 2023-11-02 03:55:21,195 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/aa3c8cd0c897447bb3b7b8258069da59_SeqId_36_ into dd77f6ddad975771f984fe41bad1dd90/family1 2023-11-02 03:55:21,195 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__ba356dqp47ubbk1gn1b8oqgm66loqhtol4np7445rp4i3kmli9qp4pfo928kg2tg/family1/eeb7343e1ed24c9da9ce96eedc45ec87 into dd77f6ddad975771f984fe41bad1dd90/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/family1/aa3c8cd0c897447bb3b7b8258069da59_SeqId_36_) 2023-11-02 03:55:21,195 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__hm12l0gmu3iaidvl76nv94qp0eh1mli9tnsbdhn439l1ao85grer2qntusgf9cq9/family1/c457d135938f4c3c9730d2de0f3bffac into b478efa9db26260ecef7476a80a8478c/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/5072a50899a94f73992578744f854b43_SeqId_34_) 2023-11-02 03:55:21,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__ba356dqp47ubbk1gn1b8oqgm66loqhtol4np7445rp4i3kmli9qp4pfo928kg2tg/family1/eeb7343e1ed24c9da9ce96eedc45ec87 2023-11-02 03:55:21,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testCompactionWithHFileLink__hm12l0gmu3iaidvl76nv94qp0eh1mli9tnsbdhn439l1ao85grer2qntusgf9cq9/family1/c457d135938f4c3c9730d2de0f3bffac 2023-11-02 03:55:21,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-02 03:55:21,198 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231102 table='testCompactionWithHFileLink' column='family1' 2023-11-02 03:55:21,198 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:21,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-11-02 03:55:21,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(1912): b478efa9db26260ecef7476a80a8478c/family1 is initiating minor compaction (all files) 2023-11-02 03:55:21,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@75120ac4; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-11-02 03:55:21,200 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202311021cc42c22257a40eb9c5aeb2784b3ad66 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202311021cc42c22257a40eb9c5aeb2784b3ad66 2023-11-02 03:55:21,201 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231102342f97ac7df94b6d87dfc0a98523bbb9 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20231102342f97ac7df94b6d87dfc0a98523bbb9 2023-11-02 03:55:21,201 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231102 table='testCompactionWithHFileLink' column='family1' 2023-11-02 03:55:21,202 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:21,202 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202311024ec670ce8f05437389ed33e14c71a7da to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202311024ec670ce8f05437389ed33e14c71a7da 2023-11-02 03:55:21,203 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202311026c16c5c49f724abdae7b6b32320f65ff to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202311026c16c5c49f724abdae7b6b32320f65ff 2023-11-02 03:55:21,204 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-02 03:55:21,204 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20231102' 2023-11-02 03:55:21,204 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202311020b8bff03978d4222a97f41a288302e66 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202311020b8bff03978d4222a97f41a288302e66 2023-11-02 03:55:21,204 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20231102 for table='testCompactionWithHFileLink' and column='family1' 2023-11-02 03:55:21,205 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023110210ea62b1fd9c46d38b55f31359a0d585 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023110210ea62b1fd9c46d38b55f31359a0d585 2023-11-02 03:55:21,207 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202311022eecb97c58c94d5c88d39499e2c15dd6 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202311022eecb97c58c94d5c88d39499e2c15dd6 2023-11-02 03:55:21,209 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202311023beda44052714bfba055af33c67ecdf6 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202311023beda44052714bfba055af33c67ecdf6 2023-11-02 03:55:21,209 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-11-02 03:55:21,209 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20231102' 2023-11-02 03:55:21,209 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231102 for table='testCompactionWithHFileLink' and column='family1' 2023-11-02 03:55:21,209 INFO [Listener at localhost/43647] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-11-02 03:55:21,212 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-02 03:55:21,218 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-02 03:55:21,222 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-02 03:55:21,223 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-02 03:55:21,231 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-02 03:55:21,232 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-02 03:55:21,255 INFO [Listener at localhost/43647] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=292 (was 239) - Thread LEAK? -, OpenFileDescriptor=796 (was 622) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=97 (was 123), ProcessCount=167 (was 170), AvailableMemoryMB=11010 (was 11081) 2023-11-02 03:55:21,272 INFO [Listener at localhost/43647] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=291, OpenFileDescriptor=796, MaxFileDescriptor=60000, SystemLoadAverage=97, ProcessCount=167, AvailableMemoryMB=11007 2023-11-02 03:55:21,278 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.HMaster$15(3014): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-11-02 03:55:21,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-11-02 03:55:21,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=104 2023-11-02 03:55:21,285 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-11-02 03:55:21,288 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 9 msec 2023-11-02 03:55:21,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=104 2023-11-02 03:55:21,385 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-11-02 03:55:21,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-11-02 03:55:21,388 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-11-02 03:55:21,388 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-11-02 03:55:21,389 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-02 03:55:21,389 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-02 03:55:21,391 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e 2023-11-02 03:55:21,391 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d 2023-11-02 03:55:21,391 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342 2023-11-02 03:55:21,391 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d empty. 2023-11-02 03:55:21,391 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342 empty. 2023-11-02 03:55:21,392 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d 2023-11-02 03:55:21,392 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e empty. 2023-11-02 03:55:21,392 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342 2023-11-02 03:55:21,392 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e 2023-11-02 03:55:21,392 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-11-02 03:55:21,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-02 03:55:21,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-02 03:55:21,799 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-11-02 03:55:21,801 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7675): creating {ENCODED => 2d76cfa810f9f3428afc40975d277f1d, NAME => 'ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp 2023-11-02 03:55:21,801 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7675): creating {ENCODED => 2b1e8f467362f169103ba63e90e10342, NAME => 'ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp 2023-11-02 03:55:21,801 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7675): creating {ENCODED => 9a8949a1c626d50be08da97ed0a2c69e, NAME => 'ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp 2023-11-02 03:55:21,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-02 03:55:22,109 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family1/c530b9cc5dc74cfc948dfc0bb07a24ae as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/c530b9cc5dc74cfc948dfc0bb07a24ae 2023-11-02 03:55:22,113 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 6ec4530a62d461a9c6949db814d37fd3/family1 of 6ec4530a62d461a9c6949db814d37fd3 into c530b9cc5dc74cfc948dfc0bb07a24ae(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-02 03:55:22,113 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6ec4530a62d461a9c6949db814d37fd3: 2023-11-02 03:55:22,113 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1698897320811; duration=1sec 2023-11-02 03:55:22,114 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-11-02 03:55:22,114 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of b478efa9db26260ecef7476a80a8478c/family1 in testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:55:22,114 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-9ed21d911efc44148a847bf3f3b4cd9b, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-656ab245fd0743afa9d05799e1b639e2, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-203e58c390f74338b4434e5c6a543c55, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-b36e505a69df415fa703d2406810ee79, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/5072a50899a94f73992578744f854b43_SeqId_34_] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp, totalSize=28.9 K 2023-11-02 03:55:22,114 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-9ed21d911efc44148a847bf3f3b4cd9b, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897297924 2023-11-02 03:55:22,115 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-656ab245fd0743afa9d05799e1b639e2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897299714 2023-11-02 03:55:22,116 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-203e58c390f74338b4434e5c6a543c55, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897301515 2023-11-02 03:55:22,116 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-b36e505a69df415fa703d2406810ee79, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897303309 2023-11-02 03:55:22,117 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 5072a50899a94f73992578744f854b43_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1698897297924 2023-11-02 03:55:22,120 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/.tmp/family2/d2049ec6bcd74b8cb5d5c365baeab354 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/d2049ec6bcd74b8cb5d5c365baeab354 2023-11-02 03:55:22,124 INFO [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 6ec4530a62d461a9c6949db814d37fd3/family2 of 6ec4530a62d461a9c6949db814d37fd3 into d2049ec6bcd74b8cb5d5c365baeab354(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-02 03:55:22,125 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6ec4530a62d461a9c6949db814d37fd3: 2023-11-02 03:55:22,125 INFO [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3., storeName=6ec4530a62d461a9c6949db814d37fd3/family2, priority=12, startTime=1698897320061; duration=1sec 2023-11-02 03:55:22,125 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-11-02 03:55:22,125 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 5 compacting, 0 eligible, 16 blocking 2023-11-02 03:55:22,125 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-11-02 03:55:22,125 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-11-02 03:55:22,125 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. because compaction request was cancelled 2023-11-02 03:55:22,125 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-11-02 03:55:22,128 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-11-02 03:55:22,128 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.HStore(1912): b478efa9db26260ecef7476a80a8478c/family2 is initiating minor compaction (all files) 2023-11-02 03:55:22,128 INFO [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.HRegion(2259): Starting compaction of b478efa9db26260ecef7476a80a8478c/family2 in testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:55:22,128 INFO [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-d59a2530e24141fd86350985f46c5be0, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-861b63499b404c75822e982990e9f0c2, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-f850896837b04a078722c50a1c779311, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-cd29ec0b5df94911b3d52331056aa7eb] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp, totalSize=20.4 K 2023-11-02 03:55:22,129 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-d59a2530e24141fd86350985f46c5be0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897297924 2023-11-02 03:55:22,130 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-861b63499b404c75822e982990e9f0c2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897299714 2023-11-02 03:55:22,131 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-f850896837b04a078722c50a1c779311, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897301515 2023-11-02 03:55:22,131 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-cd29ec0b5df94911b3d52331056aa7eb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897303309 2023-11-02 03:55:22,134 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] throttle.PressureAwareThroughputController(145): b478efa9db26260ecef7476a80a8478c#family1#compaction#117 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-02 03:55:22,147 INFO [RS:0;jenkins-hbase4:36627-longCompactions-0] throttle.PressureAwareThroughputController(145): b478efa9db26260ecef7476a80a8478c#family2#compaction#118 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-11-02 03:55:22,207 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:55:22,207 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1604): Closing 2d76cfa810f9f3428afc40975d277f1d, disabling compactions & flushes 2023-11-02 03:55:22,207 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:55:22,207 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:55:22,207 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. after waiting 0 ms 2023-11-02 03:55:22,207 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:55:22,207 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:55:22,207 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1558): Region close journal for 2d76cfa810f9f3428afc40975d277f1d: 2023-11-02 03:55:22,207 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:55:22,207 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:55:22,207 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1604): Closing 2b1e8f467362f169103ba63e90e10342, disabling compactions & flushes 2023-11-02 03:55:22,207 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1604): Closing 9a8949a1c626d50be08da97ed0a2c69e, disabling compactions & flushes 2023-11-02 03:55:22,207 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:55:22,207 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:55:22,207 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:55:22,208 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:55:22,208 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. after waiting 0 ms 2023-11-02 03:55:22,208 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. after waiting 0 ms 2023-11-02 03:55:22,208 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:55:22,208 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:55:22,208 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:55:22,208 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:55:22,208 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1558): Region close journal for 2b1e8f467362f169103ba63e90e10342: 2023-11-02 03:55:22,208 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1558): Region close journal for 9a8949a1c626d50be08da97ed0a2c69e: 2023-11-02 03:55:22,210 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-11-02 03:55:22,211 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1698897322211"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897322211"}]},"ts":"1698897322211"} 2023-11-02 03:55:22,212 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1698897322211"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897322211"}]},"ts":"1698897322211"} 2023-11-02 03:55:22,212 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1698897322211"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897322211"}]},"ts":"1698897322211"} 2023-11-02 03:55:22,218 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-11-02 03:55:22,218 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-14f9f1322bc6452cadb436583a718478, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-0830c54a581e4ef6bb871baea529ded2, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-a1705e018d254143ae0583e020fae5bc, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-b1ddcc0e1aa7400bb8a5255f06753d0a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/dbc04f93e9da408fbd7fc483661f964d_SeqId_34_] to archive 2023-11-02 03:55:22,219 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-02 03:55:22,219 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:22,219 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897322219"}]},"ts":"1698897322219"} 2023-11-02 03:55:22,220 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-11-02 03:55:22,221 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-14f9f1322bc6452cadb436583a718478 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-14f9f1322bc6452cadb436583a718478 2023-11-02 03:55:22,222 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-0830c54a581e4ef6bb871baea529ded2 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-0830c54a581e4ef6bb871baea529ded2 2023-11-02 03:55:22,223 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-a1705e018d254143ae0583e020fae5bc to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-a1705e018d254143ae0583e020fae5bc 2023-11-02 03:55:22,224 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-b1ddcc0e1aa7400bb8a5255f06753d0a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-b1ddcc0e1aa7400bb8a5255f06753d0a 2023-11-02 03:55:22,227 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/dbc04f93e9da408fbd7fc483661f964d_SeqId_34_ to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family1/dbc04f93e9da408fbd7fc483661f964d_SeqId_34_ 2023-11-02 03:55:22,231 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9a8949a1c626d50be08da97ed0a2c69e, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=2d76cfa810f9f3428afc40975d277f1d, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=2b1e8f467362f169103ba63e90e10342, ASSIGN}] 2023-11-02 03:55:22,235 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-e86f23de8c964b3f8b559674fc99287f, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-c68ad7afbd894381b90008227dceb20a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-1af22c20f93447fb9a8a6c49fdd642ff, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-bf61b264b68a4a5dacd4f9a9803c0106] to archive 2023-11-02 03:55:22,235 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=2b1e8f467362f169103ba63e90e10342, ASSIGN 2023-11-02 03:55:22,236 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9a8949a1c626d50be08da97ed0a2c69e, ASSIGN 2023-11-02 03:55:22,236 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=2d76cfa810f9f3428afc40975d277f1d, ASSIGN 2023-11-02 03:55:22,236 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:22,236 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=2b1e8f467362f169103ba63e90e10342, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=false 2023-11-02 03:55:22,236 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9a8949a1c626d50be08da97ed0a2c69e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=false 2023-11-02 03:55:22,236 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=2d76cfa810f9f3428afc40975d277f1d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=false 2023-11-02 03:55:22,238 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-e86f23de8c964b3f8b559674fc99287f to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-e86f23de8c964b3f8b559674fc99287f 2023-11-02 03:55:22,239 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-c68ad7afbd894381b90008227dceb20a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-c68ad7afbd894381b90008227dceb20a 2023-11-02 03:55:22,240 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-1af22c20f93447fb9a8a6c49fdd642ff to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-1af22c20f93447fb9a8a6c49fdd642ff 2023-11-02 03:55:22,241 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-bf61b264b68a4a5dacd4f9a9803c0106 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/family2/testCompactionWithHFileLink=6ec4530a62d461a9c6949db814d37fd3-bf61b264b68a4a5dacd4f9a9803c0106 2023-11-02 03:55:22,388 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=2d76cfa810f9f3428afc40975d277f1d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:22,388 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=2b1e8f467362f169103ba63e90e10342, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:22,388 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=9a8949a1c626d50be08da97ed0a2c69e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:22,389 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1698897322388"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897322388"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897322388"}]},"ts":"1698897322388"} 2023-11-02 03:55:22,389 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1698897322388"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897322388"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897322388"}]},"ts":"1698897322388"} 2023-11-02 03:55:22,389 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1698897322388"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897322388"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897322388"}]},"ts":"1698897322388"} 2023-11-02 03:55:22,390 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=109, ppid=107, state=RUNNABLE; OpenRegionProcedure 2d76cfa810f9f3428afc40975d277f1d, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:55:22,390 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=110, ppid=108, state=RUNNABLE; OpenRegionProcedure 2b1e8f467362f169103ba63e90e10342, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:55:22,391 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=111, ppid=106, state=RUNNABLE; OpenRegionProcedure 9a8949a1c626d50be08da97ed0a2c69e, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:55:22,493 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-02 03:55:22,541 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:22,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:55:22,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9a8949a1c626d50be08da97ed0a2c69e, NAME => 'ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e.', STARTKEY => '', ENDKEY => '1'} 2023-11-02 03:55:22,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 9a8949a1c626d50be08da97ed0a2c69e 2023-11-02 03:55:22,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:55:22,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 9a8949a1c626d50be08da97ed0a2c69e 2023-11-02 03:55:22,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 9a8949a1c626d50be08da97ed0a2c69e 2023-11-02 03:55:22,545 INFO [StoreOpener-9a8949a1c626d50be08da97ed0a2c69e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9a8949a1c626d50be08da97ed0a2c69e 2023-11-02 03:55:22,546 DEBUG [StoreOpener-9a8949a1c626d50be08da97ed0a2c69e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1 2023-11-02 03:55:22,546 DEBUG [StoreOpener-9a8949a1c626d50be08da97ed0a2c69e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1 2023-11-02 03:55:22,547 INFO [StoreOpener-9a8949a1c626d50be08da97ed0a2c69e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9a8949a1c626d50be08da97ed0a2c69e columnFamilyName family1 2023-11-02 03:55:22,548 INFO [StoreOpener-9a8949a1c626d50be08da97ed0a2c69e-1] regionserver.HStore(310): Store=9a8949a1c626d50be08da97ed0a2c69e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:55:22,548 INFO [StoreOpener-9a8949a1c626d50be08da97ed0a2c69e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 9a8949a1c626d50be08da97ed0a2c69e 2023-11-02 03:55:22,549 DEBUG [StoreOpener-9a8949a1c626d50be08da97ed0a2c69e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2 2023-11-02 03:55:22,550 DEBUG [StoreOpener-9a8949a1c626d50be08da97ed0a2c69e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2 2023-11-02 03:55:22,550 INFO [StoreOpener-9a8949a1c626d50be08da97ed0a2c69e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9a8949a1c626d50be08da97ed0a2c69e columnFamilyName family2 2023-11-02 03:55:22,551 INFO [StoreOpener-9a8949a1c626d50be08da97ed0a2c69e-1] regionserver.HStore(310): Store=9a8949a1c626d50be08da97ed0a2c69e/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:55:22,552 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e 2023-11-02 03:55:22,552 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e 2023-11-02 03:55:22,553 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-02 03:55:22,554 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 9a8949a1c626d50be08da97ed0a2c69e 2023-11-02 03:55:22,555 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:55:22,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 9a8949a1c626d50be08da97ed0a2c69e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59979983, jitterRate=-0.10622860491275787}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-02 03:55:22,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 9a8949a1c626d50be08da97ed0a2c69e: 2023-11-02 03:55:22,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e., pid=111, masterSystemTime=1698897322541 2023-11-02 03:55:22,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:55:22,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:55:22,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:55:22,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2d76cfa810f9f3428afc40975d277f1d, NAME => 'ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d.', STARTKEY => '1', ENDKEY => '2'} 2023-11-02 03:55:22,558 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=9a8949a1c626d50be08da97ed0a2c69e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:22,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 2d76cfa810f9f3428afc40975d277f1d 2023-11-02 03:55:22,558 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1698897322558"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897322558"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897322558"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897322558"}]},"ts":"1698897322558"} 2023-11-02 03:55:22,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:55:22,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2d76cfa810f9f3428afc40975d277f1d 2023-11-02 03:55:22,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2d76cfa810f9f3428afc40975d277f1d 2023-11-02 03:55:22,559 INFO [StoreOpener-2d76cfa810f9f3428afc40975d277f1d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2d76cfa810f9f3428afc40975d277f1d 2023-11-02 03:55:22,560 DEBUG [StoreOpener-2d76cfa810f9f3428afc40975d277f1d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1 2023-11-02 03:55:22,560 DEBUG [StoreOpener-2d76cfa810f9f3428afc40975d277f1d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1 2023-11-02 03:55:22,560 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=111, resume processing ppid=106 2023-11-02 03:55:22,560 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=111, ppid=106, state=SUCCESS; OpenRegionProcedure 9a8949a1c626d50be08da97ed0a2c69e, server=jenkins-hbase4.apache.org,36627,1698897201574 in 168 msec 2023-11-02 03:55:22,561 INFO [StoreOpener-2d76cfa810f9f3428afc40975d277f1d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2d76cfa810f9f3428afc40975d277f1d columnFamilyName family1 2023-11-02 03:55:22,561 INFO [StoreOpener-2d76cfa810f9f3428afc40975d277f1d-1] regionserver.HStore(310): Store=2d76cfa810f9f3428afc40975d277f1d/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:55:22,561 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=9a8949a1c626d50be08da97ed0a2c69e, ASSIGN in 332 msec 2023-11-02 03:55:22,562 INFO [StoreOpener-2d76cfa810f9f3428afc40975d277f1d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 2d76cfa810f9f3428afc40975d277f1d 2023-11-02 03:55:22,563 DEBUG [StoreOpener-2d76cfa810f9f3428afc40975d277f1d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2 2023-11-02 03:55:22,563 DEBUG [StoreOpener-2d76cfa810f9f3428afc40975d277f1d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2 2023-11-02 03:55:22,563 INFO [StoreOpener-2d76cfa810f9f3428afc40975d277f1d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2d76cfa810f9f3428afc40975d277f1d columnFamilyName family2 2023-11-02 03:55:22,564 INFO [StoreOpener-2d76cfa810f9f3428afc40975d277f1d-1] regionserver.HStore(310): Store=2d76cfa810f9f3428afc40975d277f1d/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:55:22,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d 2023-11-02 03:55:22,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d 2023-11-02 03:55:22,566 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-02 03:55:22,567 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2d76cfa810f9f3428afc40975d277f1d 2023-11-02 03:55:22,569 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:55:22,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2d76cfa810f9f3428afc40975d277f1d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63802296, jitterRate=-0.04927170276641846}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-02 03:55:22,569 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2d76cfa810f9f3428afc40975d277f1d: 2023-11-02 03:55:22,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d., pid=109, masterSystemTime=1698897322541 2023-11-02 03:55:22,570 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:55:22,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:55:22,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:55:22,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2b1e8f467362f169103ba63e90e10342, NAME => 'ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342.', STARTKEY => '2', ENDKEY => ''} 2023-11-02 03:55:22,571 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=2d76cfa810f9f3428afc40975d277f1d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:22,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 2b1e8f467362f169103ba63e90e10342 2023-11-02 03:55:22,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:55:22,571 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1698897322571"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897322571"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897322571"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897322571"}]},"ts":"1698897322571"} 2023-11-02 03:55:22,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2b1e8f467362f169103ba63e90e10342 2023-11-02 03:55:22,571 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2b1e8f467362f169103ba63e90e10342 2023-11-02 03:55:22,572 INFO [StoreOpener-2b1e8f467362f169103ba63e90e10342-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 2b1e8f467362f169103ba63e90e10342 2023-11-02 03:55:22,573 DEBUG [StoreOpener-2b1e8f467362f169103ba63e90e10342-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1 2023-11-02 03:55:22,573 DEBUG [StoreOpener-2b1e8f467362f169103ba63e90e10342-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1 2023-11-02 03:55:22,573 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=109, resume processing ppid=107 2023-11-02 03:55:22,573 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=109, ppid=107, state=SUCCESS; OpenRegionProcedure 2d76cfa810f9f3428afc40975d277f1d, server=jenkins-hbase4.apache.org,36627,1698897201574 in 182 msec 2023-11-02 03:55:22,574 INFO [StoreOpener-2b1e8f467362f169103ba63e90e10342-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2b1e8f467362f169103ba63e90e10342 columnFamilyName family1 2023-11-02 03:55:22,574 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=2d76cfa810f9f3428afc40975d277f1d, ASSIGN in 344 msec 2023-11-02 03:55:22,574 INFO [StoreOpener-2b1e8f467362f169103ba63e90e10342-1] regionserver.HStore(310): Store=2b1e8f467362f169103ba63e90e10342/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:55:22,575 INFO [StoreOpener-2b1e8f467362f169103ba63e90e10342-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 2b1e8f467362f169103ba63e90e10342 2023-11-02 03:55:22,576 DEBUG [StoreOpener-2b1e8f467362f169103ba63e90e10342-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2 2023-11-02 03:55:22,576 DEBUG [StoreOpener-2b1e8f467362f169103ba63e90e10342-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2 2023-11-02 03:55:22,576 INFO [StoreOpener-2b1e8f467362f169103ba63e90e10342-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2b1e8f467362f169103ba63e90e10342 columnFamilyName family2 2023-11-02 03:55:22,577 INFO [StoreOpener-2b1e8f467362f169103ba63e90e10342-1] regionserver.HStore(310): Store=2b1e8f467362f169103ba63e90e10342/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:55:22,578 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342 2023-11-02 03:55:22,578 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342 2023-11-02 03:55:22,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-02 03:55:22,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2b1e8f467362f169103ba63e90e10342 2023-11-02 03:55:22,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:55:22,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2b1e8f467362f169103ba63e90e10342; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72603818, jitterRate=0.08188119530677795}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-02 03:55:22,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2b1e8f467362f169103ba63e90e10342: 2023-11-02 03:55:22,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342., pid=110, masterSystemTime=1698897322541 2023-11-02 03:55:22,584 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:55:22,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:55:22,584 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=2b1e8f467362f169103ba63e90e10342, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:22,584 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1698897322584"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897322584"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897322584"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897322584"}]},"ts":"1698897322584"} 2023-11-02 03:55:22,587 INFO [PEWorker-3] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=110, resume processing ppid=108 2023-11-02 03:55:22,587 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=110, ppid=108, state=SUCCESS; OpenRegionProcedure 2b1e8f467362f169103ba63e90e10342, server=jenkins-hbase4.apache.org,36627,1698897201574 in 196 msec 2023-11-02 03:55:22,588 INFO [PEWorker-5] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=108, resume processing ppid=105 2023-11-02 03:55:22,588 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=2b1e8f467362f169103ba63e90e10342, ASSIGN in 358 msec 2023-11-02 03:55:22,588 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-02 03:55:22,589 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897322589"}]},"ts":"1698897322589"} 2023-11-02 03:55:22,589 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-11-02 03:55:22,592 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-11-02 03:55:22,593 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.2060 sec 2023-11-02 03:55:22,953 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.15384615384615385, tune throughput to 57.69 MB/second 2023-11-02 03:55:22,956 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family1/45ee6a01a98c406b936eabb498a5b38e as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/45ee6a01a98c406b936eabb498a5b38e 2023-11-02 03:55:22,960 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in b478efa9db26260ecef7476a80a8478c/family1 of b478efa9db26260ecef7476a80a8478c into 45ee6a01a98c406b936eabb498a5b38e(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-02 03:55:22,960 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for b478efa9db26260ecef7476a80a8478c: 2023-11-02 03:55:22,960 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1698897321199; duration=0sec 2023-11-02 03:55:22,961 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-11-02 03:55:22,961 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-11-02 03:55:22,961 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-11-02 03:55:22,961 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-11-02 03:55:22,961 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. because compaction request was cancelled 2023-11-02 03:55:22,967 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/.tmp/family2/29e460fa7f3f4a8ca7732ce28b138709 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/29e460fa7f3f4a8ca7732ce28b138709 2023-11-02 03:55:22,971 INFO [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in b478efa9db26260ecef7476a80a8478c/family2 of b478efa9db26260ecef7476a80a8478c into 29e460fa7f3f4a8ca7732ce28b138709(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-02 03:55:22,971 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for b478efa9db26260ecef7476a80a8478c: 2023-11-02 03:55:22,971 INFO [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c., storeName=b478efa9db26260ecef7476a80a8478c/family2, priority=12, startTime=1698897320126; duration=0sec 2023-11-02 03:55:22,971 DEBUG [RS:0;jenkins-hbase4:36627-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:55:23,003 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-9ed21d911efc44148a847bf3f3b4cd9b, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-656ab245fd0743afa9d05799e1b639e2, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-203e58c390f74338b4434e5c6a543c55, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-b36e505a69df415fa703d2406810ee79, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/5072a50899a94f73992578744f854b43_SeqId_34_] to archive 2023-11-02 03:55:23,004 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:23,005 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-9ed21d911efc44148a847bf3f3b4cd9b to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-9ed21d911efc44148a847bf3f3b4cd9b 2023-11-02 03:55:23,006 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-656ab245fd0743afa9d05799e1b639e2 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-656ab245fd0743afa9d05799e1b639e2 2023-11-02 03:55:23,007 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-203e58c390f74338b4434e5c6a543c55 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-203e58c390f74338b4434e5c6a543c55 2023-11-02 03:55:23,008 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-b36e505a69df415fa703d2406810ee79 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-b36e505a69df415fa703d2406810ee79 2023-11-02 03:55:23,009 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/5072a50899a94f73992578744f854b43_SeqId_34_ to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family1/5072a50899a94f73992578744f854b43_SeqId_34_ 2023-11-02 03:55:23,010 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-d59a2530e24141fd86350985f46c5be0, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-861b63499b404c75822e982990e9f0c2, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-f850896837b04a078722c50a1c779311, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-cd29ec0b5df94911b3d52331056aa7eb] to archive 2023-11-02 03:55:23,011 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:23,012 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-d59a2530e24141fd86350985f46c5be0 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-d59a2530e24141fd86350985f46c5be0 2023-11-02 03:55:23,013 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-861b63499b404c75822e982990e9f0c2 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-861b63499b404c75822e982990e9f0c2 2023-11-02 03:55:23,013 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-f850896837b04a078722c50a1c779311 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-f850896837b04a078722c50a1c779311 2023-11-02 03:55:23,014 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-cd29ec0b5df94911b3d52331056aa7eb to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/family2/testCompactionWithHFileLink=b478efa9db26260ecef7476a80a8478c-cd29ec0b5df94911b3d52331056aa7eb 2023-11-02 03:55:23,215 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-02 03:55:23,407 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-11-02 03:55:23,494 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-11-02 03:55:23,494 INFO [Listener at localhost/43647] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-11-02 03:55:23,499 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:23,500 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:23,500 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:23,502 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:23,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-11-02 03:55:23,504 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-11-02 03:55:23,504 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:23,504 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-11-02 03:55:23,504 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-11-02 03:55:23,504 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:23,504 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:23,505 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:23,505 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:23,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:23,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:23,506 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:23,506 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:23,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:23,506 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:23,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:23,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:23,507 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-11-02 03:55:23,507 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-11-02 03:55:23,507 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-11-02 03:55:23,507 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:23,507 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-11-02 03:55:23,507 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:55:23,507 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:55:23,507 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. started... 2023-11-02 03:55:23,507 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. started... 2023-11-02 03:55:23,508 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-02 03:55:23,509 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:23,509 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:55:23,509 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b1e8f467362f169103ba63e90e10342 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-02 03:55:23,509 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 9a8949a1c626d50be08da97ed0a2c69e 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-02 03:55:23,509 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. started... 2023-11-02 03:55:23,510 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 2d76cfa810f9f3428afc40975d277f1d 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-02 03:55:23,924 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311026a6980bf4f404a4d8e8ac192e65fada1 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202311026a6980bf4f404a4d8e8ac192e65fada1 2023-11-02 03:55:23,924 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311021a30e740b3584f0881edde43c58b3fb6 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202311021a30e740b3584f0881edde43c58b3fb6 2023-11-02 03:55:23,924 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231102dbdca19fc46643619123e6de1fa16d3f to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20231102dbdca19fc46643619123e6de1fa16d3f 2023-11-02 03:55:24,335 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family1/491c6d5bd1604af7b3b8614bc2fa7c91 2023-11-02 03:55:24,346 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family1/e8023142a6bd4a54b5155f7b0c15b4e8 2023-11-02 03:55:24,348 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family1/1c0de0fd651c43829e32f9fc2db5db44 2023-11-02 03:55:24,749 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023110283d1fcbcf03f47348227da146ba239d2 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023110283d1fcbcf03f47348227da146ba239d2 2023-11-02 03:55:24,759 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311023754013a290d4a86b6d2e66841b6c0b1 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202311023754013a290d4a86b6d2e66841b6c0b1 2023-11-02 03:55:24,760 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231102d3ecc62cc1f643c98a25b24a460ba818 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20231102d3ecc62cc1f643c98a25b24a460ba818 2023-11-02 03:55:25,153 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family2/2e3fbdecaac44e86903d093a14451ad4 2023-11-02 03:55:25,157 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family1/491c6d5bd1604af7b3b8614bc2fa7c91 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/491c6d5bd1604af7b3b8614bc2fa7c91 2023-11-02 03:55:25,161 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/491c6d5bd1604af7b3b8614bc2fa7c91, entries=4, sequenceid=5, filesize=5.4 K 2023-11-02 03:55:25,161 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family2/2e3fbdecaac44e86903d093a14451ad4 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/2e3fbdecaac44e86903d093a14451ad4 2023-11-02 03:55:25,163 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family2/0b791aeadb564a7983c5dbba1768e52f 2023-11-02 03:55:25,164 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family2/60244cc80fa249aaa45ddf4b3e30f31c 2023-11-02 03:55:25,165 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/2e3fbdecaac44e86903d093a14451ad4, entries=2, sequenceid=5, filesize=5.1 K 2023-11-02 03:55:25,166 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 2b1e8f467362f169103ba63e90e10342 in 1657ms, sequenceid=5, compaction requested=false 2023-11-02 03:55:25,166 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b1e8f467362f169103ba63e90e10342: 2023-11-02 03:55:25,166 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:55:25,167 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-02 03:55:25,167 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family1/e8023142a6bd4a54b5155f7b0c15b4e8 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/e8023142a6bd4a54b5155f7b0c15b4e8 2023-11-02 03:55:25,169 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family1/1c0de0fd651c43829e32f9fc2db5db44 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/1c0de0fd651c43829e32f9fc2db5db44 2023-11-02 03:55:25,171 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/e8023142a6bd4a54b5155f7b0c15b4e8, entries=4, sequenceid=5, filesize=5.4 K 2023-11-02 03:55:25,171 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family2/0b791aeadb564a7983c5dbba1768e52f as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/0b791aeadb564a7983c5dbba1768e52f 2023-11-02 03:55:25,172 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/1c0de0fd651c43829e32f9fc2db5db44, entries=4, sequenceid=5, filesize=5.4 K 2023-11-02 03:55:25,173 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family2/60244cc80fa249aaa45ddf4b3e30f31c as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/60244cc80fa249aaa45ddf4b3e30f31c 2023-11-02 03:55:25,175 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/0b791aeadb564a7983c5dbba1768e52f, entries=2, sequenceid=5, filesize=5.1 K 2023-11-02 03:55:25,175 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 2d76cfa810f9f3428afc40975d277f1d in 1665ms, sequenceid=5, compaction requested=false 2023-11-02 03:55:25,175 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2d76cfa810f9f3428afc40975d277f1d: 2023-11-02 03:55:25,175 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:55:25,175 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-02 03:55:25,176 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/60244cc80fa249aaa45ddf4b3e30f31c, entries=2, sequenceid=5, filesize=5.1 K 2023-11-02 03:55:25,177 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 9a8949a1c626d50be08da97ed0a2c69e in 1668ms, sequenceid=5, compaction requested=false 2023-11-02 03:55:25,177 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 9a8949a1c626d50be08da97ed0a2c69e: 2023-11-02 03:55:25,177 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:55:25,177 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-02 03:55:25,177 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-02 03:55:25,177 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:25,177 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-11-02 03:55:25,177 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-11-02 03:55:25,179 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:25,179 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:25,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:25,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:25,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:25,179 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:25,179 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:25,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:25,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:25,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:25,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:25,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:25,180 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-11-02 03:55:25,180 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5f7a220f[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:25,180 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-11-02 03:55:25,180 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:25,182 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:25,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:25,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:25,182 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-11-02 03:55:25,182 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-11-02 03:55:25,182 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:25,182 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:25,182 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:25,183 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:25,183 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-11-02 03:55:25,183 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:25,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:25,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:25,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:25,183 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-11-02 03:55:25,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:25,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:25,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:25,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:25,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:25,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:25,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:25,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:25,186 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-11-02 03:55:25,186 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-11-02 03:55:25,186 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:25,186 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:25,186 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-11-02 03:55:25,186 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:25,188 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:25,188 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:25,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:25,188 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:25,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:25,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:25,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:25,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:25,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:25,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:25,189 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:25,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:25,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:25,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:25,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:25,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:25,190 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:25,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:25,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:25,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:25,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:25,192 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:25,192 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:25,192 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:25,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:25,192 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:25,192 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-11-02 03:55:25,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:25,192 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-11-02 03:55:25,192 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:25,192 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:25,192 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:25,192 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:25,192 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:25,192 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:25,192 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:25,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:25,192 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:25,193 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:25,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:25,293 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:25,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:25,295 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:25,295 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:25,296 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:25,298 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:25,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-11-02 03:55:25,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-11-02 03:55:25,299 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-11-02 03:55:25,299 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:25,299 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-11-02 03:55:25,299 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-11-02 03:55:25,300 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:25,300 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:25,301 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:25,301 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:25,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:25,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:25,301 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:25,301 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:25,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:25,302 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:25,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:25,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:25,302 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-11-02 03:55:25,302 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-11-02 03:55:25,302 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-11-02 03:55:25,302 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-11-02 03:55:25,302 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:25,303 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-11-02 03:55:25,303 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:55:25,303 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. started... 2023-11-02 03:55:25,303 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:55:25,303 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-02 03:55:25,303 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:55:25,303 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b1e8f467362f169103ba63e90e10342 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-02 03:55:25,303 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:25,303 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. started... 2023-11-02 03:55:25,303 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. started... 2023-11-02 03:55:25,303 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 9a8949a1c626d50be08da97ed0a2c69e 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-02 03:55:25,303 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 2d76cfa810f9f3428afc40975d277f1d 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-02 03:55:25,716 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311027ba4330afb824e6e92105681af0f850d to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202311027ba4330afb824e6e92105681af0f850d 2023-11-02 03:55:25,716 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311025d9cb8634bf14e71bfaa02764a080509 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202311025d9cb8634bf14e71bfaa02764a080509 2023-11-02 03:55:25,716 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231102ce5a816ff23c4f6cb11e60a03dae7db0 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20231102ce5a816ff23c4f6cb11e60a03dae7db0 2023-11-02 03:55:26,120 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family1/323002c82d0049dc9fea1293f56049d2 2023-11-02 03:55:26,121 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family1/b977ff816aff4cac988e0ac38cbeb3de 2023-11-02 03:55:26,121 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family1/7ac27d4e831c486a89abec17844b6907 2023-11-02 03:55:26,142 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231102cf46b5ce2f0f4e21a02c7fb4b6a02a9d to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20231102cf46b5ce2f0f4e21a02c7fb4b6a02a9d 2023-11-02 03:55:26,538 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231102aaf85efcc0164fd9b03ef981cc4e28c2 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20231102aaf85efcc0164fd9b03ef981cc4e28c2 2023-11-02 03:55:26,541 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231102f8797ec3b2e04fcfa74404cafe14e8fc to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20231102f8797ec3b2e04fcfa74404cafe14e8fc 2023-11-02 03:55:26,546 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family2/63e0192554ad4a9c9ae6caaf1016d96d 2023-11-02 03:55:26,549 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family1/b977ff816aff4cac988e0ac38cbeb3de as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/b977ff816aff4cac988e0ac38cbeb3de 2023-11-02 03:55:26,552 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/b977ff816aff4cac988e0ac38cbeb3de, entries=4, sequenceid=9, filesize=5.4 K 2023-11-02 03:55:26,553 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family2/63e0192554ad4a9c9ae6caaf1016d96d as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/63e0192554ad4a9c9ae6caaf1016d96d 2023-11-02 03:55:26,556 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/63e0192554ad4a9c9ae6caaf1016d96d, entries=2, sequenceid=9, filesize=5.1 K 2023-11-02 03:55:26,557 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 2d76cfa810f9f3428afc40975d277f1d in 1254ms, sequenceid=9, compaction requested=false 2023-11-02 03:55:26,557 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2d76cfa810f9f3428afc40975d277f1d: 2023-11-02 03:55:26,557 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:55:26,557 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-02 03:55:26,943 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family2/415a27fb8aa743519b66f21376d08f01 2023-11-02 03:55:26,946 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family2/cc996b9cae2146e38c98f64641151960 2023-11-02 03:55:26,947 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family1/7ac27d4e831c486a89abec17844b6907 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/7ac27d4e831c486a89abec17844b6907 2023-11-02 03:55:26,950 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family1/323002c82d0049dc9fea1293f56049d2 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/323002c82d0049dc9fea1293f56049d2 2023-11-02 03:55:26,951 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/7ac27d4e831c486a89abec17844b6907, entries=4, sequenceid=9, filesize=5.4 K 2023-11-02 03:55:26,952 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family2/415a27fb8aa743519b66f21376d08f01 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/415a27fb8aa743519b66f21376d08f01 2023-11-02 03:55:26,953 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/323002c82d0049dc9fea1293f56049d2, entries=4, sequenceid=9, filesize=5.4 K 2023-11-02 03:55:26,954 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family2/cc996b9cae2146e38c98f64641151960 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/cc996b9cae2146e38c98f64641151960 2023-11-02 03:55:26,955 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/415a27fb8aa743519b66f21376d08f01, entries=2, sequenceid=9, filesize=5.1 K 2023-11-02 03:55:26,956 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 9a8949a1c626d50be08da97ed0a2c69e in 1653ms, sequenceid=9, compaction requested=false 2023-11-02 03:55:26,956 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 9a8949a1c626d50be08da97ed0a2c69e: 2023-11-02 03:55:26,956 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:55:26,956 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-02 03:55:26,958 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/cc996b9cae2146e38c98f64641151960, entries=2, sequenceid=9, filesize=5.1 K 2023-11-02 03:55:26,958 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 2b1e8f467362f169103ba63e90e10342 in 1655ms, sequenceid=9, compaction requested=false 2023-11-02 03:55:26,958 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b1e8f467362f169103ba63e90e10342: 2023-11-02 03:55:26,958 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:55:26,959 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-02 03:55:26,959 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-02 03:55:26,959 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:26,959 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-11-02 03:55:26,959 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-11-02 03:55:26,960 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:26,960 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:26,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:26,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:26,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:26,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:26,961 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:26,961 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:26,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:26,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:26,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:26,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:26,963 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-11-02 03:55:26,963 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3d489422[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:26,963 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-11-02 03:55:26,963 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:26,964 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:26,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:26,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:26,964 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-11-02 03:55:26,964 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-11-02 03:55:26,964 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:26,964 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:26,964 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:26,966 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:26,966 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-11-02 03:55:26,966 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:26,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:26,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:26,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:26,966 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-11-02 03:55:26,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:26,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:26,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:26,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:26,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:26,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:26,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:26,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:26,968 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-11-02 03:55:26,968 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-11-02 03:55:26,968 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:26,968 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:26,968 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-11-02 03:55:26,968 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:26,970 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:26,971 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:26,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:26,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:26,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:26,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:26,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:26,971 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:26,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:26,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:26,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:26,971 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:26,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:26,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:26,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:26,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:26,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:26,973 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:26,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:26,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:26,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:26,975 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:26,975 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:26,975 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:26,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:26,975 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:26,975 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:26,975 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:26,975 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:26,975 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:26,975 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:26,975 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-11-02 03:55:26,975 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:26,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:26,975 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:26,976 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:26,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:26,976 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:26,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:27,076 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:27,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:27,078 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:27,078 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:27,078 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:27,080 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:27,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-11-02 03:55:27,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-11-02 03:55:27,082 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-11-02 03:55:27,082 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:27,082 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-11-02 03:55:27,082 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-11-02 03:55:27,082 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:27,083 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:27,084 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:27,084 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:27,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:27,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:27,084 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:27,084 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:27,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:27,085 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:27,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:27,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:27,085 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-11-02 03:55:27,085 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-11-02 03:55:27,086 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-11-02 03:55:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-11-02 03:55:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-11-02 03:55:27,086 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:55:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-02 03:55:27,086 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:27,086 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. started... 2023-11-02 03:55:27,086 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:55:27,086 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:55:27,086 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. started... 2023-11-02 03:55:27,086 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b1e8f467362f169103ba63e90e10342 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-02 03:55:27,086 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. started... 2023-11-02 03:55:27,086 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 2d76cfa810f9f3428afc40975d277f1d 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-02 03:55:27,086 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 9a8949a1c626d50be08da97ed0a2c69e 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-02 03:55:27,111 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231102bdf2fbbe5dd94938a30e5d05a6d80890 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20231102bdf2fbbe5dd94938a30e5d05a6d80890 2023-11-02 03:55:27,115 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family1/629b70e41522407c8c258af55ec15739 2023-11-02 03:55:27,125 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311022e3228b85241421cb31c31cb1d57f306 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202311022e3228b85241421cb31c31cb1d57f306 2023-11-02 03:55:27,130 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family2/1ad1ba6f40ae41999cea86ebd85e1a6c 2023-11-02 03:55:27,133 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family1/629b70e41522407c8c258af55ec15739 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/629b70e41522407c8c258af55ec15739 2023-11-02 03:55:27,136 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/629b70e41522407c8c258af55ec15739, entries=4, sequenceid=13, filesize=5.4 K 2023-11-02 03:55:27,137 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family2/1ad1ba6f40ae41999cea86ebd85e1a6c as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/1ad1ba6f40ae41999cea86ebd85e1a6c 2023-11-02 03:55:27,142 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/1ad1ba6f40ae41999cea86ebd85e1a6c, entries=2, sequenceid=13, filesize=5.1 K 2023-11-02 03:55:27,142 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 2d76cfa810f9f3428afc40975d277f1d in 56ms, sequenceid=13, compaction requested=true 2023-11-02 03:55:27,143 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2d76cfa810f9f3428afc40975d277f1d: 2023-11-02 03:55:27,143 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:55:27,143 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-02 03:55:27,505 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231102daf8f0a98ada4c5198b4ca96afe6fb8d to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20231102daf8f0a98ada4c5198b4ca96afe6fb8d 2023-11-02 03:55:27,506 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311024bdc1e8540ec4476b999e71e5aff5b51 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202311024bdc1e8540ec4476b999e71e5aff5b51 2023-11-02 03:55:27,510 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family1/2e0af6533cd84f65b5e6ddeb54e72039 2023-11-02 03:55:27,520 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311020425649dae924e72a4fe084b75ff641c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202311020425649dae924e72a4fe084b75ff641c 2023-11-02 03:55:27,524 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family2/fb7ddc2e8bd44d16842331722f58cc8b 2023-11-02 03:55:27,527 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family1/2e0af6533cd84f65b5e6ddeb54e72039 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/2e0af6533cd84f65b5e6ddeb54e72039 2023-11-02 03:55:27,530 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/2e0af6533cd84f65b5e6ddeb54e72039, entries=4, sequenceid=13, filesize=5.4 K 2023-11-02 03:55:27,531 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family2/fb7ddc2e8bd44d16842331722f58cc8b as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/fb7ddc2e8bd44d16842331722f58cc8b 2023-11-02 03:55:27,533 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/fb7ddc2e8bd44d16842331722f58cc8b, entries=2, sequenceid=13, filesize=5.1 K 2023-11-02 03:55:27,534 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 2b1e8f467362f169103ba63e90e10342 in 448ms, sequenceid=13, compaction requested=true 2023-11-02 03:55:27,534 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b1e8f467362f169103ba63e90e10342: 2023-11-02 03:55:27,534 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:55:27,534 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-02 03:55:27,911 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family1/28cffc12d25e4bcabb100dd0ea1285c7 2023-11-02 03:55:28,328 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311025b0e72875a314feebce037f18de86182 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202311025b0e72875a314feebce037f18de86182 2023-11-02 03:55:28,334 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family2/cae9712ebb58441ab1f1fa7acc090731 2023-11-02 03:55:28,338 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family1/28cffc12d25e4bcabb100dd0ea1285c7 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/28cffc12d25e4bcabb100dd0ea1285c7 2023-11-02 03:55:28,341 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/28cffc12d25e4bcabb100dd0ea1285c7, entries=4, sequenceid=13, filesize=5.4 K 2023-11-02 03:55:28,342 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family2/cae9712ebb58441ab1f1fa7acc090731 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/cae9712ebb58441ab1f1fa7acc090731 2023-11-02 03:55:28,346 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/cae9712ebb58441ab1f1fa7acc090731, entries=2, sequenceid=13, filesize=5.1 K 2023-11-02 03:55:28,347 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 9a8949a1c626d50be08da97ed0a2c69e in 1260ms, sequenceid=13, compaction requested=true 2023-11-02 03:55:28,347 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 9a8949a1c626d50be08da97ed0a2c69e: 2023-11-02 03:55:28,347 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:55:28,347 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-02 03:55:28,347 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-02 03:55:28,347 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:28,347 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-11-02 03:55:28,347 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-11-02 03:55:28,348 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:28,348 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:28,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:28,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:28,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:28,349 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:28,349 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:28,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:28,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:28,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:28,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:28,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:28,350 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-11-02 03:55:28,350 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15d070cc[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:28,350 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-11-02 03:55:28,350 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:28,351 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:28,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:28,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:28,352 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-11-02 03:55:28,352 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-11-02 03:55:28,352 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:28,352 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:28,352 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:28,353 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-11-02 03:55:28,353 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:28,353 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:28,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:28,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:28,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:28,353 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-11-02 03:55:28,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:28,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:28,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:28,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:28,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:28,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:28,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:28,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:28,356 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-11-02 03:55:28,356 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-11-02 03:55:28,356 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:28,356 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:28,356 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-11-02 03:55:28,356 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:28,359 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:28,359 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:28,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:28,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:28,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:28,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:28,359 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:28,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:28,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:28,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:28,362 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:28,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:28,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:28,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:28,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:28,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:28,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:28,363 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:28,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:28,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:28,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:28,384 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:28,384 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:28,384 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:28,384 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:28,384 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:28,384 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:28,384 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:28,384 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:28,384 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-11-02 03:55:28,384 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:28,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:28,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:28,384 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:28,385 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:28,385 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:28,385 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:28,385 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-02 03:55:28,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:28,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:28,385 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x345a3dae to 127.0.0.1:65185 2023-11-02 03:55:28,385 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:55:28,386 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-02 03:55:28,386 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7a1f1eb7 to 127.0.0.1:65185 2023-11-02 03:55:28,386 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:55:28,485 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:28,486 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:28,487 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:28,487 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:28,488 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HRegion(8805): writing data to region ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:28,490 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:28,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-11-02 03:55:28,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-11-02 03:55:28,491 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-11-02 03:55:28,492 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:28,492 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-11-02 03:55:28,492 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-11-02 03:55:28,492 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:28,492 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:28,493 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:28,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:28,494 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:28,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:28,494 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:28,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:28,494 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:28,494 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:28,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:28,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:28,495 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-11-02 03:55:28,495 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-11-02 03:55:28,495 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-11-02 03:55:28,495 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:28,495 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-11-02 03:55:28,495 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:55:28,495 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. started... 2023-11-02 03:55:28,496 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b1e8f467362f169103ba63e90e10342 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-02 03:55:28,496 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:55:28,496 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-02 03:55:28,496 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. started... 2023-11-02 03:55:28,496 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:55:28,496 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:28,496 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 9a8949a1c626d50be08da97ed0a2c69e 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-02 03:55:28,496 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. started... 2023-11-02 03:55:28,496 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 2d76cfa810f9f3428afc40975d277f1d 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-02 03:55:28,508 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311027d732c5008d041a0b08596f2e3162903 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202311027d732c5008d041a0b08596f2e3162903 2023-11-02 03:55:28,509 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231102e7266818ca5740d88a45819ee5e0d05b to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20231102e7266818ca5740d88a45819ee5e0d05b 2023-11-02 03:55:28,513 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family1/f1879999210a47858882dd21cdeacb3f 2023-11-02 03:55:28,514 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family1/b563f1caff1f4ba480baead5500e9bf5 2023-11-02 03:55:28,910 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311027fc0601db6574d0385d09b7addf2f20f to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202311027fc0601db6574d0385d09b7addf2f20f 2023-11-02 03:55:28,930 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231102380728fb087641e8ad6a6f3d586e45b3 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20231102380728fb087641e8ad6a6f3d586e45b3 2023-11-02 03:55:28,931 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231102d9c47a9ed0264bdd8efea68d63dd4391 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20231102d9c47a9ed0264bdd8efea68d63dd4391 2023-11-02 03:55:29,316 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family1/2967a26db4194fdebc0084e21208f6be 2023-11-02 03:55:29,336 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family2/a489d52bbbe44683a9cc641978b277da 2023-11-02 03:55:29,340 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family1/b563f1caff1f4ba480baead5500e9bf5 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/b563f1caff1f4ba480baead5500e9bf5 2023-11-02 03:55:29,341 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family2/33be771ac6a242cca99a24fb62f78d42 2023-11-02 03:55:29,346 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/b563f1caff1f4ba480baead5500e9bf5, entries=4, sequenceid=17, filesize=5.4 K 2023-11-02 03:55:29,347 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family2/a489d52bbbe44683a9cc641978b277da as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/a489d52bbbe44683a9cc641978b277da 2023-11-02 03:55:29,347 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family1/f1879999210a47858882dd21cdeacb3f as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/f1879999210a47858882dd21cdeacb3f 2023-11-02 03:55:29,351 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/f1879999210a47858882dd21cdeacb3f, entries=4, sequenceid=17, filesize=5.4 K 2023-11-02 03:55:29,352 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/a489d52bbbe44683a9cc641978b277da, entries=2, sequenceid=17, filesize=5.1 K 2023-11-02 03:55:29,352 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family2/33be771ac6a242cca99a24fb62f78d42 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/33be771ac6a242cca99a24fb62f78d42 2023-11-02 03:55:29,353 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 2d76cfa810f9f3428afc40975d277f1d in 857ms, sequenceid=17, compaction requested=true 2023-11-02 03:55:29,353 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2d76cfa810f9f3428afc40975d277f1d: 2023-11-02 03:55:29,353 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:55:29,353 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-02 03:55:29,356 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/33be771ac6a242cca99a24fb62f78d42, entries=2, sequenceid=17, filesize=5.1 K 2023-11-02 03:55:29,357 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 9a8949a1c626d50be08da97ed0a2c69e in 861ms, sequenceid=17, compaction requested=true 2023-11-02 03:55:29,357 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 9a8949a1c626d50be08da97ed0a2c69e: 2023-11-02 03:55:29,357 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:55:29,357 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-02 03:55:29,733 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023110209e19c2c7630436c910b7bab9bf827e4 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023110209e19c2c7630436c910b7bab9bf827e4 2023-11-02 03:55:29,738 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family2/7222a4c37ea74c8fbe5e89c0b33ca616 2023-11-02 03:55:29,742 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family1/2967a26db4194fdebc0084e21208f6be as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/2967a26db4194fdebc0084e21208f6be 2023-11-02 03:55:29,746 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/2967a26db4194fdebc0084e21208f6be, entries=4, sequenceid=17, filesize=5.4 K 2023-11-02 03:55:29,747 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family2/7222a4c37ea74c8fbe5e89c0b33ca616 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/7222a4c37ea74c8fbe5e89c0b33ca616 2023-11-02 03:55:29,750 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/7222a4c37ea74c8fbe5e89c0b33ca616, entries=2, sequenceid=17, filesize=5.1 K 2023-11-02 03:55:29,751 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 2b1e8f467362f169103ba63e90e10342 in 1256ms, sequenceid=17, compaction requested=true 2023-11-02 03:55:29,751 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b1e8f467362f169103ba63e90e10342: 2023-11-02 03:55:29,751 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:55:29,751 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-02 03:55:29,751 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-02 03:55:29,751 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:29,752 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-11-02 03:55:29,752 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-11-02 03:55:29,753 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:29,753 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:29,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:29,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:29,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:29,754 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:29,754 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:29,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:29,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:29,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:29,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:29,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:29,755 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-11-02 03:55:29,755 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@453a6caf[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:29,755 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-11-02 03:55:29,756 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:29,757 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:29,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:29,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:29,757 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-11-02 03:55:29,757 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-11-02 03:55:29,757 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:29,757 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:29,757 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:29,760 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-11-02 03:55:29,760 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:29,760 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:29,760 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-11-02 03:55:29,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:29,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:29,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:29,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:29,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:29,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:29,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:29,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:29,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:29,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:29,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:29,762 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-11-02 03:55:29,762 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-11-02 03:55:29,762 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:29,762 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:29,762 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-11-02 03:55:29,762 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:29,764 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:29,764 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:29,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:29,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:29,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:29,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:29,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:29,764 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:29,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:29,765 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:29,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:29,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:29,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:29,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:29,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:29,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:29,766 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:29,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:29,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:29,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:29,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:29,770 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:29,770 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:29,770 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:29,770 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:29,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:29,770 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:29,770 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:29,770 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-11-02 03:55:29,770 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:29,770 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:29,770 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-11-02 03:55:29,770 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:29,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:29,770 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:29,770 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:29,771 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:29,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:29,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:29,771 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:29,871 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:29,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:29,887 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-02 03:55:29,899 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-02 03:55:29,910 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-02 03:55:29,913 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-02 03:55:29,965 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:29,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-11-02 03:55:29,967 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-11-02 03:55:29,967 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-11-02 03:55:29,967 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:29,967 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-11-02 03:55:29,967 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-11-02 03:55:29,967 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:29,968 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:29,969 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:29,969 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:29,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:29,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:29,969 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:29,969 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:29,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:29,970 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:29,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:29,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:29,970 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-11-02 03:55:29,971 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-11-02 03:55:29,971 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-11-02 03:55:29,971 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-11-02 03:55:29,971 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:29,971 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-11-02 03:55:29,971 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:55:29,971 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:55:29,971 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. started... 2023-11-02 03:55:29,971 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. started... 2023-11-02 03:55:29,971 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 9a8949a1c626d50be08da97ed0a2c69e 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-02 03:55:29,971 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2b1e8f467362f169103ba63e90e10342 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-02 03:55:29,973 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-02 03:55:29,973 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:29,973 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:55:29,973 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. started... 2023-11-02 03:55:29,973 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 2d76cfa810f9f3428afc40975d277f1d 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-02 03:55:30,010 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family1/80418441b42643028111da1e1c0a1b50 2023-11-02 03:55:30,014 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 80418441b42643028111da1e1c0a1b50 2023-11-02 03:55:30,026 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family2/ee219804b4a94a7387813023300268d7 2023-11-02 03:55:30,030 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ee219804b4a94a7387813023300268d7 2023-11-02 03:55:30,030 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family1/80418441b42643028111da1e1c0a1b50 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/80418441b42643028111da1e1c0a1b50 2023-11-02 03:55:30,033 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 80418441b42643028111da1e1c0a1b50 2023-11-02 03:55:30,034 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/80418441b42643028111da1e1c0a1b50, entries=3, sequenceid=23, filesize=5.0 K 2023-11-02 03:55:30,034 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family2/ee219804b4a94a7387813023300268d7 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/ee219804b4a94a7387813023300268d7 2023-11-02 03:55:30,038 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ee219804b4a94a7387813023300268d7 2023-11-02 03:55:30,038 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/ee219804b4a94a7387813023300268d7, entries=1, sequenceid=23, filesize=5.0 K 2023-11-02 03:55:30,039 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 9a8949a1c626d50be08da97ed0a2c69e in 68ms, sequenceid=23, compaction requested=true 2023-11-02 03:55:30,039 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 9a8949a1c626d50be08da97ed0a2c69e: 2023-11-02 03:55:30,039 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:55:30,039 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-02 03:55:30,810 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family1/d686fe110cc04c29bbeb0c83a68b123c 2023-11-02 03:55:30,817 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family1/f12e3ab269d0451daf3eec127632ff16 2023-11-02 03:55:30,819 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d686fe110cc04c29bbeb0c83a68b123c 2023-11-02 03:55:30,821 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f12e3ab269d0451daf3eec127632ff16 2023-11-02 03:55:31,236 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family2/64689f4d70d84bbfb859d4f09df70c8f 2023-11-02 03:55:31,238 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family2/4d7ed8b47b324af88f76281c6d37f4a3 2023-11-02 03:55:31,240 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 64689f4d70d84bbfb859d4f09df70c8f 2023-11-02 03:55:31,240 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family1/d686fe110cc04c29bbeb0c83a68b123c as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/d686fe110cc04c29bbeb0c83a68b123c 2023-11-02 03:55:31,242 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4d7ed8b47b324af88f76281c6d37f4a3 2023-11-02 03:55:31,243 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family1/f12e3ab269d0451daf3eec127632ff16 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/f12e3ab269d0451daf3eec127632ff16 2023-11-02 03:55:31,244 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d686fe110cc04c29bbeb0c83a68b123c 2023-11-02 03:55:31,245 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/d686fe110cc04c29bbeb0c83a68b123c, entries=3, sequenceid=23, filesize=5.0 K 2023-11-02 03:55:31,246 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family2/64689f4d70d84bbfb859d4f09df70c8f as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/64689f4d70d84bbfb859d4f09df70c8f 2023-11-02 03:55:31,246 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f12e3ab269d0451daf3eec127632ff16 2023-11-02 03:55:31,246 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/f12e3ab269d0451daf3eec127632ff16, entries=3, sequenceid=23, filesize=5.0 K 2023-11-02 03:55:31,247 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family2/4d7ed8b47b324af88f76281c6d37f4a3 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/4d7ed8b47b324af88f76281c6d37f4a3 2023-11-02 03:55:31,250 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 64689f4d70d84bbfb859d4f09df70c8f 2023-11-02 03:55:31,250 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/64689f4d70d84bbfb859d4f09df70c8f, entries=1, sequenceid=23, filesize=5.0 K 2023-11-02 03:55:31,251 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4d7ed8b47b324af88f76281c6d37f4a3 2023-11-02 03:55:31,251 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 2b1e8f467362f169103ba63e90e10342 in 1280ms, sequenceid=23, compaction requested=true 2023-11-02 03:55:31,251 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/4d7ed8b47b324af88f76281c6d37f4a3, entries=1, sequenceid=23, filesize=5.0 K 2023-11-02 03:55:31,251 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2b1e8f467362f169103ba63e90e10342: 2023-11-02 03:55:31,251 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:55:31,251 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-02 03:55:31,252 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 2d76cfa810f9f3428afc40975d277f1d in 1279ms, sequenceid=23, compaction requested=true 2023-11-02 03:55:31,252 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 2d76cfa810f9f3428afc40975d277f1d: 2023-11-02 03:55:31,252 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:55:31,252 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-02 03:55:31,252 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-02 03:55:31,252 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:31,252 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-11-02 03:55:31,252 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-11-02 03:55:31,254 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:31,254 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:31,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:31,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:31,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:31,255 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:31,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:31,255 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:31,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:31,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:31,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:31,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:31,257 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-11-02 03:55:31,257 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4bde445d[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:31,257 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-11-02 03:55:31,257 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:31,258 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:31,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:31,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:31,258 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-11-02 03:55:31,258 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-11-02 03:55:31,258 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:31,259 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:31,259 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:31,260 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-11-02 03:55:31,260 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:31,260 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:31,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:31,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:31,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:31,260 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-11-02 03:55:31,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:31,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:31,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:31,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:31,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:31,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:31,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:31,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:31,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-11-02 03:55:31,262 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-11-02 03:55:31,263 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:31,263 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:31,263 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-11-02 03:55:31,263 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:31,264 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:31,264 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:31,264 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:31,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:31,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:31,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:31,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:31,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:31,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:31,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:31,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:31,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:31,268 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:31,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:31,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:31,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:31,270 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:31,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:31,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:31,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-11-02 03:55:31,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:31,273 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:31,273 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:31,273 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:31,273 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:31,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:31,273 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:31,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:31,273 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-11-02 03:55:31,273 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-11-02 03:55:31,273 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-11-02 03:55:31,273 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:31,273 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:31,273 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:31,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:31,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:31,273 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-11-02 03:55:31,274 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-11-02 03:55:31,274 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:31,274 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:31,374 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:31,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:31,375 DEBUG [Listener at localhost/43647] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-02 03:55:31,376 DEBUG [Listener at localhost/43647] regionserver.HStore(1912): 2b1e8f467362f169103ba63e90e10342/family1 is initiating major compaction (all files) 2023-11-02 03:55:31,376 INFO [Listener at localhost/43647] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-02 03:55:31,376 INFO [Listener at localhost/43647] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-02 03:55:31,376 INFO [Listener at localhost/43647] regionserver.HRegion(2259): Starting compaction of 2b1e8f467362f169103ba63e90e10342/family1 in ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:55:31,376 INFO [Listener at localhost/43647] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/491c6d5bd1604af7b3b8614bc2fa7c91, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/323002c82d0049dc9fea1293f56049d2, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/2e0af6533cd84f65b5e6ddeb54e72039, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/2967a26db4194fdebc0084e21208f6be, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/d686fe110cc04c29bbeb0c83a68b123c] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp, totalSize=26.6 K 2023-11-02 03:55:31,377 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 491c6d5bd1604af7b3b8614bc2fa7c91, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897323500 2023-11-02 03:55:31,377 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 323002c82d0049dc9fea1293f56049d2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897325296 2023-11-02 03:55:31,377 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 2e0af6533cd84f65b5e6ddeb54e72039, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897327078 2023-11-02 03:55:31,378 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 2967a26db4194fdebc0084e21208f6be, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897328488 2023-11-02 03:55:31,378 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting d686fe110cc04c29bbeb0c83a68b123c, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-02 03:55:31,388 INFO [Listener at localhost/43647] throttle.PressureAwareThroughputController(145): 2b1e8f467362f169103ba63e90e10342#family1#compaction#149 average throughput is 0.55 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:55:31,405 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023110231450bfec249457aa0d123b580b5ebea_del 2023-11-02 03:55:31,405 INFO [Listener at localhost/43647] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023110231450bfec249457aa0d123b580b5ebea_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023110231450bfec249457aa0d123b580b5ebea_del 2023-11-02 03:55:31,420 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 49a35c57a80d4c5b986e15d1cbd3d00b 2023-11-02 03:55:31,421 DEBUG [Listener at localhost/43647] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family1/49a35c57a80d4c5b986e15d1cbd3d00b as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/49a35c57a80d4c5b986e15d1cbd3d00b 2023-11-02 03:55:31,424 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 49a35c57a80d4c5b986e15d1cbd3d00b 2023-11-02 03:55:31,425 INFO [Listener at localhost/43647] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 2b1e8f467362f169103ba63e90e10342/family1 of 2b1e8f467362f169103ba63e90e10342 into 49a35c57a80d4c5b986e15d1cbd3d00b(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-02 03:55:31,425 DEBUG [Listener at localhost/43647] regionserver.HRegion(2289): Compaction status journal for 2b1e8f467362f169103ba63e90e10342: 2023-11-02 03:55:31,425 DEBUG [Listener at localhost/43647] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-02 03:55:31,425 DEBUG [Listener at localhost/43647] regionserver.HStore(1912): 2b1e8f467362f169103ba63e90e10342/family2 is initiating major compaction (all files) 2023-11-02 03:55:31,425 INFO [Listener at localhost/43647] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-02 03:55:31,425 INFO [Listener at localhost/43647] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-02 03:55:31,425 INFO [Listener at localhost/43647] regionserver.HRegion(2259): Starting compaction of 2b1e8f467362f169103ba63e90e10342/family2 in ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:55:31,425 INFO [Listener at localhost/43647] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/2e3fbdecaac44e86903d093a14451ad4, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/cc996b9cae2146e38c98f64641151960, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/fb7ddc2e8bd44d16842331722f58cc8b, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/7222a4c37ea74c8fbe5e89c0b33ca616, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/64689f4d70d84bbfb859d4f09df70c8f] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp, totalSize=25.3 K 2023-11-02 03:55:31,426 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 2e3fbdecaac44e86903d093a14451ad4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897323500 2023-11-02 03:55:31,426 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting cc996b9cae2146e38c98f64641151960, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897325296 2023-11-02 03:55:31,427 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting fb7ddc2e8bd44d16842331722f58cc8b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897327078 2023-11-02 03:55:31,427 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 7222a4c37ea74c8fbe5e89c0b33ca616, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897328488 2023-11-02 03:55:31,427 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 64689f4d70d84bbfb859d4f09df70c8f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-02 03:55:31,442 INFO [Listener at localhost/43647] throttle.PressureAwareThroughputController(145): 2b1e8f467362f169103ba63e90e10342#family2#compaction#150 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:55:31,449 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202311029e9bfab403674d9097fb833ce2392f8b_del 2023-11-02 03:55:31,449 INFO [Listener at localhost/43647] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311029e9bfab403674d9097fb833ce2392f8b_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202311029e9bfab403674d9097fb833ce2392f8b_del 2023-11-02 03:55:31,511 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/491c6d5bd1604af7b3b8614bc2fa7c91, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/323002c82d0049dc9fea1293f56049d2, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/2e0af6533cd84f65b5e6ddeb54e72039, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/2967a26db4194fdebc0084e21208f6be, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/d686fe110cc04c29bbeb0c83a68b123c] to archive 2023-11-02 03:55:31,511 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:31,513 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/491c6d5bd1604af7b3b8614bc2fa7c91 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/491c6d5bd1604af7b3b8614bc2fa7c91 2023-11-02 03:55:31,514 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/323002c82d0049dc9fea1293f56049d2 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/323002c82d0049dc9fea1293f56049d2 2023-11-02 03:55:31,515 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/2e0af6533cd84f65b5e6ddeb54e72039 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/2e0af6533cd84f65b5e6ddeb54e72039 2023-11-02 03:55:31,516 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/2967a26db4194fdebc0084e21208f6be to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/2967a26db4194fdebc0084e21208f6be 2023-11-02 03:55:31,517 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/d686fe110cc04c29bbeb0c83a68b123c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/d686fe110cc04c29bbeb0c83a68b123c 2023-11-02 03:55:31,861 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 393f77cca3c84c1982b9812392c5f3cd 2023-11-02 03:55:31,862 DEBUG [Listener at localhost/43647] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/.tmp/family2/393f77cca3c84c1982b9812392c5f3cd as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/393f77cca3c84c1982b9812392c5f3cd 2023-11-02 03:55:31,865 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 393f77cca3c84c1982b9812392c5f3cd 2023-11-02 03:55:31,866 INFO [Listener at localhost/43647] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 2b1e8f467362f169103ba63e90e10342/family2 of 2b1e8f467362f169103ba63e90e10342 into 393f77cca3c84c1982b9812392c5f3cd(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-02 03:55:31,866 DEBUG [Listener at localhost/43647] regionserver.HRegion(2289): Compaction status journal for 2b1e8f467362f169103ba63e90e10342: 2023-11-02 03:55:31,866 DEBUG [Listener at localhost/43647] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-02 03:55:31,867 DEBUG [Listener at localhost/43647] regionserver.HStore(1912): 9a8949a1c626d50be08da97ed0a2c69e/family1 is initiating major compaction (all files) 2023-11-02 03:55:31,867 INFO [Listener at localhost/43647] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-02 03:55:31,867 INFO [Listener at localhost/43647] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-02 03:55:31,867 INFO [Listener at localhost/43647] regionserver.HRegion(2259): Starting compaction of 9a8949a1c626d50be08da97ed0a2c69e/family1 in ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:55:31,867 INFO [Listener at localhost/43647] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/1c0de0fd651c43829e32f9fc2db5db44, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/7ac27d4e831c486a89abec17844b6907, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/28cffc12d25e4bcabb100dd0ea1285c7, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/f1879999210a47858882dd21cdeacb3f, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/80418441b42643028111da1e1c0a1b50] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp, totalSize=26.6 K 2023-11-02 03:55:31,867 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 1c0de0fd651c43829e32f9fc2db5db44, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897323499 2023-11-02 03:55:31,868 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 7ac27d4e831c486a89abec17844b6907, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897325295 2023-11-02 03:55:31,868 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 28cffc12d25e4bcabb100dd0ea1285c7, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897327078 2023-11-02 03:55:31,868 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting f1879999210a47858882dd21cdeacb3f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897328487 2023-11-02 03:55:31,869 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 80418441b42643028111da1e1c0a1b50, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-02 03:55:31,876 INFO [Listener at localhost/43647] throttle.PressureAwareThroughputController(145): 9a8949a1c626d50be08da97ed0a2c69e#family1#compaction#151 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:55:31,906 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/2e3fbdecaac44e86903d093a14451ad4, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/cc996b9cae2146e38c98f64641151960, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/fb7ddc2e8bd44d16842331722f58cc8b, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/7222a4c37ea74c8fbe5e89c0b33ca616, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/64689f4d70d84bbfb859d4f09df70c8f] to archive 2023-11-02 03:55:31,907 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:31,908 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/2e3fbdecaac44e86903d093a14451ad4 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/2e3fbdecaac44e86903d093a14451ad4 2023-11-02 03:55:31,909 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/cc996b9cae2146e38c98f64641151960 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/cc996b9cae2146e38c98f64641151960 2023-11-02 03:55:31,909 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/fb7ddc2e8bd44d16842331722f58cc8b to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/fb7ddc2e8bd44d16842331722f58cc8b 2023-11-02 03:55:31,910 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/7222a4c37ea74c8fbe5e89c0b33ca616 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/7222a4c37ea74c8fbe5e89c0b33ca616 2023-11-02 03:55:31,911 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/64689f4d70d84bbfb859d4f09df70c8f to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family2/64689f4d70d84bbfb859d4f09df70c8f 2023-11-02 03:55:32,283 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023110203795142ef33455fb2bcc07dc9d0d743_del 2023-11-02 03:55:32,283 INFO [Listener at localhost/43647] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023110203795142ef33455fb2bcc07dc9d0d743_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023110203795142ef33455fb2bcc07dc9d0d743_del 2023-11-02 03:55:33,098 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2d4baf74465e466285001b54cf0c4531 2023-11-02 03:55:33,099 DEBUG [Listener at localhost/43647] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family1/2d4baf74465e466285001b54cf0c4531 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/2d4baf74465e466285001b54cf0c4531 2023-11-02 03:55:33,103 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2d4baf74465e466285001b54cf0c4531 2023-11-02 03:55:33,104 INFO [Listener at localhost/43647] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 9a8949a1c626d50be08da97ed0a2c69e/family1 of 9a8949a1c626d50be08da97ed0a2c69e into 2d4baf74465e466285001b54cf0c4531(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-02 03:55:33,104 DEBUG [Listener at localhost/43647] regionserver.HRegion(2289): Compaction status journal for 9a8949a1c626d50be08da97ed0a2c69e: 2023-11-02 03:55:33,104 DEBUG [Listener at localhost/43647] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-02 03:55:33,104 DEBUG [Listener at localhost/43647] regionserver.HStore(1912): 9a8949a1c626d50be08da97ed0a2c69e/family2 is initiating major compaction (all files) 2023-11-02 03:55:33,104 INFO [Listener at localhost/43647] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-02 03:55:33,104 INFO [Listener at localhost/43647] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-02 03:55:33,104 INFO [Listener at localhost/43647] regionserver.HRegion(2259): Starting compaction of 9a8949a1c626d50be08da97ed0a2c69e/family2 in ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:55:33,105 INFO [Listener at localhost/43647] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/60244cc80fa249aaa45ddf4b3e30f31c, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/415a27fb8aa743519b66f21376d08f01, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/cae9712ebb58441ab1f1fa7acc090731, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/33be771ac6a242cca99a24fb62f78d42, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/ee219804b4a94a7387813023300268d7] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp, totalSize=25.3 K 2023-11-02 03:55:33,105 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 60244cc80fa249aaa45ddf4b3e30f31c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897323499 2023-11-02 03:55:33,105 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 415a27fb8aa743519b66f21376d08f01, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897325295 2023-11-02 03:55:33,106 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting cae9712ebb58441ab1f1fa7acc090731, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897327078 2023-11-02 03:55:33,106 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 33be771ac6a242cca99a24fb62f78d42, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897328487 2023-11-02 03:55:33,106 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting ee219804b4a94a7387813023300268d7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-02 03:55:33,115 INFO [Listener at localhost/43647] throttle.PressureAwareThroughputController(145): 9a8949a1c626d50be08da97ed0a2c69e#family2#compaction#152 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:55:33,203 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/1c0de0fd651c43829e32f9fc2db5db44, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/7ac27d4e831c486a89abec17844b6907, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/28cffc12d25e4bcabb100dd0ea1285c7, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/f1879999210a47858882dd21cdeacb3f, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/80418441b42643028111da1e1c0a1b50] to archive 2023-11-02 03:55:33,204 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:33,206 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/1c0de0fd651c43829e32f9fc2db5db44 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/1c0de0fd651c43829e32f9fc2db5db44 2023-11-02 03:55:33,207 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/7ac27d4e831c486a89abec17844b6907 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/7ac27d4e831c486a89abec17844b6907 2023-11-02 03:55:33,208 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/28cffc12d25e4bcabb100dd0ea1285c7 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/28cffc12d25e4bcabb100dd0ea1285c7 2023-11-02 03:55:33,209 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/f1879999210a47858882dd21cdeacb3f to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/f1879999210a47858882dd21cdeacb3f 2023-11-02 03:55:33,210 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/80418441b42643028111da1e1c0a1b50 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/80418441b42643028111da1e1c0a1b50 2023-11-02 03:55:33,523 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023110282e853db376d455bb06b35fb3325c48d_del 2023-11-02 03:55:33,523 INFO [Listener at localhost/43647] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023110282e853db376d455bb06b35fb3325c48d_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023110282e853db376d455bb06b35fb3325c48d_del 2023-11-02 03:55:34,336 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 17454604686c4379a51426fd067c2802 2023-11-02 03:55:34,337 DEBUG [Listener at localhost/43647] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/.tmp/family2/17454604686c4379a51426fd067c2802 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/17454604686c4379a51426fd067c2802 2023-11-02 03:55:34,342 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 17454604686c4379a51426fd067c2802 2023-11-02 03:55:34,343 INFO [Listener at localhost/43647] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 9a8949a1c626d50be08da97ed0a2c69e/family2 of 9a8949a1c626d50be08da97ed0a2c69e into 17454604686c4379a51426fd067c2802(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-02 03:55:34,343 DEBUG [Listener at localhost/43647] regionserver.HRegion(2289): Compaction status journal for 9a8949a1c626d50be08da97ed0a2c69e: 2023-11-02 03:55:34,343 DEBUG [Listener at localhost/43647] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-02 03:55:34,343 DEBUG [Listener at localhost/43647] regionserver.HStore(1912): 2d76cfa810f9f3428afc40975d277f1d/family1 is initiating major compaction (all files) 2023-11-02 03:55:34,343 INFO [Listener at localhost/43647] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-02 03:55:34,343 INFO [Listener at localhost/43647] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-02 03:55:34,343 INFO [Listener at localhost/43647] regionserver.HRegion(2259): Starting compaction of 2d76cfa810f9f3428afc40975d277f1d/family1 in ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:55:34,343 INFO [Listener at localhost/43647] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/e8023142a6bd4a54b5155f7b0c15b4e8, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/b977ff816aff4cac988e0ac38cbeb3de, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/629b70e41522407c8c258af55ec15739, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/b563f1caff1f4ba480baead5500e9bf5, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/f12e3ab269d0451daf3eec127632ff16] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp, totalSize=26.6 K 2023-11-02 03:55:34,344 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting e8023142a6bd4a54b5155f7b0c15b4e8, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897323500 2023-11-02 03:55:34,344 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting b977ff816aff4cac988e0ac38cbeb3de, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897325295 2023-11-02 03:55:34,344 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 629b70e41522407c8c258af55ec15739, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897327078 2023-11-02 03:55:34,345 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting b563f1caff1f4ba480baead5500e9bf5, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897328487 2023-11-02 03:55:34,345 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting f12e3ab269d0451daf3eec127632ff16, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-02 03:55:34,354 INFO [Listener at localhost/43647] throttle.PressureAwareThroughputController(145): 2d76cfa810f9f3428afc40975d277f1d#family1#compaction#153 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:55:34,365 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202311024e3ed871f408409a8de919763463fbd4_del 2023-11-02 03:55:34,366 INFO [Listener at localhost/43647] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311024e3ed871f408409a8de919763463fbd4_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202311024e3ed871f408409a8de919763463fbd4_del 2023-11-02 03:55:34,403 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/60244cc80fa249aaa45ddf4b3e30f31c, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/415a27fb8aa743519b66f21376d08f01, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/cae9712ebb58441ab1f1fa7acc090731, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/33be771ac6a242cca99a24fb62f78d42, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/ee219804b4a94a7387813023300268d7] to archive 2023-11-02 03:55:34,405 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:34,406 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/60244cc80fa249aaa45ddf4b3e30f31c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/60244cc80fa249aaa45ddf4b3e30f31c 2023-11-02 03:55:34,407 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/415a27fb8aa743519b66f21376d08f01 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/415a27fb8aa743519b66f21376d08f01 2023-11-02 03:55:34,408 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/cae9712ebb58441ab1f1fa7acc090731 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/cae9712ebb58441ab1f1fa7acc090731 2023-11-02 03:55:34,409 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/33be771ac6a242cca99a24fb62f78d42 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/33be771ac6a242cca99a24fb62f78d42 2023-11-02 03:55:34,410 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/ee219804b4a94a7387813023300268d7 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family2/ee219804b4a94a7387813023300268d7 2023-11-02 03:55:35,179 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63a994cb360744b283422d968c94be92 2023-11-02 03:55:35,179 DEBUG [Listener at localhost/43647] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family1/63a994cb360744b283422d968c94be92 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/63a994cb360744b283422d968c94be92 2023-11-02 03:55:35,183 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63a994cb360744b283422d968c94be92 2023-11-02 03:55:35,183 INFO [Listener at localhost/43647] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 2d76cfa810f9f3428afc40975d277f1d/family1 of 2d76cfa810f9f3428afc40975d277f1d into 63a994cb360744b283422d968c94be92(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-02 03:55:35,184 DEBUG [Listener at localhost/43647] regionserver.HRegion(2289): Compaction status journal for 2d76cfa810f9f3428afc40975d277f1d: 2023-11-02 03:55:35,184 DEBUG [Listener at localhost/43647] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-02 03:55:35,184 DEBUG [Listener at localhost/43647] regionserver.HStore(1912): 2d76cfa810f9f3428afc40975d277f1d/family2 is initiating major compaction (all files) 2023-11-02 03:55:35,184 INFO [Listener at localhost/43647] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-02 03:55:35,184 INFO [Listener at localhost/43647] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-02 03:55:35,184 INFO [Listener at localhost/43647] regionserver.HRegion(2259): Starting compaction of 2d76cfa810f9f3428afc40975d277f1d/family2 in ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:55:35,184 INFO [Listener at localhost/43647] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/0b791aeadb564a7983c5dbba1768e52f, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/63e0192554ad4a9c9ae6caaf1016d96d, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/1ad1ba6f40ae41999cea86ebd85e1a6c, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/a489d52bbbe44683a9cc641978b277da, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/4d7ed8b47b324af88f76281c6d37f4a3] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp, totalSize=25.3 K 2023-11-02 03:55:35,184 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 0b791aeadb564a7983c5dbba1768e52f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897323500 2023-11-02 03:55:35,185 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 63e0192554ad4a9c9ae6caaf1016d96d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897325295 2023-11-02 03:55:35,185 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 1ad1ba6f40ae41999cea86ebd85e1a6c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897327078 2023-11-02 03:55:35,185 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting a489d52bbbe44683a9cc641978b277da, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897328487 2023-11-02 03:55:35,186 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 4d7ed8b47b324af88f76281c6d37f4a3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-02 03:55:35,194 INFO [Listener at localhost/43647] throttle.PressureAwareThroughputController(145): 2d76cfa810f9f3428afc40975d277f1d#family2#compaction#154 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:55:35,204 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/e8023142a6bd4a54b5155f7b0c15b4e8, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/b977ff816aff4cac988e0ac38cbeb3de, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/629b70e41522407c8c258af55ec15739, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/b563f1caff1f4ba480baead5500e9bf5, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/f12e3ab269d0451daf3eec127632ff16] to archive 2023-11-02 03:55:35,205 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:35,206 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/e8023142a6bd4a54b5155f7b0c15b4e8 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/e8023142a6bd4a54b5155f7b0c15b4e8 2023-11-02 03:55:35,207 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/b977ff816aff4cac988e0ac38cbeb3de to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/b977ff816aff4cac988e0ac38cbeb3de 2023-11-02 03:55:35,208 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/629b70e41522407c8c258af55ec15739 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/629b70e41522407c8c258af55ec15739 2023-11-02 03:55:35,209 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/b563f1caff1f4ba480baead5500e9bf5 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/b563f1caff1f4ba480baead5500e9bf5 2023-11-02 03:55:35,210 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/f12e3ab269d0451daf3eec127632ff16 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/f12e3ab269d0451daf3eec127632ff16 2023-11-02 03:55:35,609 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231102e85395aa1af04ea7a9d93dfe9c925891_del 2023-11-02 03:55:35,609 INFO [Listener at localhost/43647] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231102e85395aa1af04ea7a9d93dfe9c925891_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20231102e85395aa1af04ea7a9d93dfe9c925891_del 2023-11-02 03:55:36,423 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a9f5a1ae0e934d8eafd39610485e30df 2023-11-02 03:55:36,424 DEBUG [Listener at localhost/43647] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/.tmp/family2/a9f5a1ae0e934d8eafd39610485e30df as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/a9f5a1ae0e934d8eafd39610485e30df 2023-11-02 03:55:36,429 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a9f5a1ae0e934d8eafd39610485e30df 2023-11-02 03:55:36,430 INFO [Listener at localhost/43647] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 2d76cfa810f9f3428afc40975d277f1d/family2 of 2d76cfa810f9f3428afc40975d277f1d into a9f5a1ae0e934d8eafd39610485e30df(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-02 03:55:36,430 DEBUG [Listener at localhost/43647] regionserver.HRegion(2289): Compaction status journal for 2d76cfa810f9f3428afc40975d277f1d: 2023-11-02 03:55:36,434 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-02 03:55:36,437 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-02 03:55:36,440 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-02 03:55:36,441 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-02 03:55:36,445 INFO [Listener at localhost/43647] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-02 03:55:36,459 INFO [Listener at localhost/43647] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-11-02 03:55:36,461 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023110203795142ef33455fb2bcc07dc9d0d743_del 2023-11-02 03:55:36,464 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202311024e3ed871f408409a8de919763463fbd4_del 2023-11-02 03:55:36,467 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c2023110231450bfec249457aa0d123b580b5ebea_del 2023-11-02 03:55:36,467 INFO [Listener at localhost/43647] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-11-02 03:55:36,468 DEBUG [Listener at localhost/43647] zookeeper.ReadOnlyZKClient(139): Connect 0x5f35c3cc to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:55:36,473 DEBUG [Listener at localhost/43647] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@277b31ca, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:55:36,474 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231102 for table='ns:testMinorCompaction' and column='family1' 2023-11-02 03:55:36,474 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231102 for table='ns:testMinorCompaction' and column='family1' 2023-11-02 03:55:36,474 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20231102 for table='ns:testMinorCompaction' and column='family1' 2023-11-02 03:55:36,501 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023110216354205303e4ccfb24360fc967e0e8a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023110216354205303e4ccfb24360fc967e0e8a 2023-11-02 03:55:36,502 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231102 table='ns:testMinorCompaction' column='family1' 2023-11-02 03:55:36,502 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231102382d4260ba60490e8baafbcadbd84632 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20231102382d4260ba60490e8baafbcadbd84632 2023-11-02 03:55:36,503 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231102 table='ns:testMinorCompaction' column='family1' 2023-11-02 03:55:36,504 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/0b791aeadb564a7983c5dbba1768e52f, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/63e0192554ad4a9c9ae6caaf1016d96d, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/1ad1ba6f40ae41999cea86ebd85e1a6c, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/a489d52bbbe44683a9cc641978b277da, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/4d7ed8b47b324af88f76281c6d37f4a3] to archive 2023-11-02 03:55:36,504 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:36,506 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/0b791aeadb564a7983c5dbba1768e52f to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/0b791aeadb564a7983c5dbba1768e52f 2023-11-02 03:55:36,507 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/63e0192554ad4a9c9ae6caaf1016d96d to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/63e0192554ad4a9c9ae6caaf1016d96d 2023-11-02 03:55:36,508 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/1ad1ba6f40ae41999cea86ebd85e1a6c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/1ad1ba6f40ae41999cea86ebd85e1a6c 2023-11-02 03:55:36,510 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/a489d52bbbe44683a9cc641978b277da to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/a489d52bbbe44683a9cc641978b277da 2023-11-02 03:55:36,511 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/4d7ed8b47b324af88f76281c6d37f4a3 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family2/4d7ed8b47b324af88f76281c6d37f4a3 2023-11-02 03:55:36,515 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-02 03:55:36,520 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57584, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-02 03:55:36,526 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-02 03:55:36,529 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36712, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-02 03:55:36,547 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231102/family1/84861aa4b833475093277afa35adab69 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-11-02 03:55:36,548 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231102/family1/84861aa4b833475093277afa35adab69}] 2023-11-02 03:55:36,548 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231102/family1/d456748798ab49d3962210861ec78fb3 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-11-02 03:55:36,550 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231102/family1/d456748798ab49d3962210861ec78fb3}] 2023-11-02 03:55:36,556 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231102/family1/84861aa4b833475093277afa35adab69 for inclusion in 9a8949a1c626d50be08da97ed0a2c69e/family1 2023-11-02 03:55:36,556 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231102/family1/d456748798ab49d3962210861ec78fb3 for inclusion in 2d76cfa810f9f3428afc40975d277f1d/family1 2023-11-02 03:55:36,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-11-02 03:55:36,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last=1 2023-11-02 03:55:36,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for 9a8949a1c626d50be08da97ed0a2c69e: 2023-11-02 03:55:36,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231102/family1/84861aa4b833475093277afa35adab69 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__ns_testMinorCompaction__bg5598r89hvsjpem36135690mie5n7cqb90bdf21lul5p00j7eehgq4ahldroche/family1/84861aa4b833475093277afa35adab69 2023-11-02 03:55:36,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-11-02 03:55:36,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first=1 last=2 2023-11-02 03:55:36,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for 2d76cfa810f9f3428afc40975d277f1d: 2023-11-02 03:55:36,561 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231102/family1/d456748798ab49d3962210861ec78fb3 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__ns_testMinorCompaction__nb7ol65h1t8po034ij9e7rgn0d4p1njifho0uk2ka2kdrs7fe54dkda4p2if749h/family1/d456748798ab49d3962210861ec78fb3 2023-11-02 03:55:36,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__ns_testMinorCompaction__bg5598r89hvsjpem36135690mie5n7cqb90bdf21lul5p00j7eehgq4ahldroche/family1/84861aa4b833475093277afa35adab69 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/9771d52a7de949b1b2fa1b31b5cb5e20_SeqId_28_ 2023-11-02 03:55:36,562 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__ns_testMinorCompaction__bg5598r89hvsjpem36135690mie5n7cqb90bdf21lul5p00j7eehgq4ahldroche/family1/84861aa4b833475093277afa35adab69 into 9a8949a1c626d50be08da97ed0a2c69e/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/9771d52a7de949b1b2fa1b31b5cb5e20_SeqId_28_ - updating store file list. 2023-11-02 03:55:36,563 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__ns_testMinorCompaction__nb7ol65h1t8po034ij9e7rgn0d4p1njifho0uk2ka2kdrs7fe54dkda4p2if749h/family1/d456748798ab49d3962210861ec78fb3 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/d3449e498baa449aa7836512381a8f77_SeqId_28_ 2023-11-02 03:55:36,564 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__ns_testMinorCompaction__nb7ol65h1t8po034ij9e7rgn0d4p1njifho0uk2ka2kdrs7fe54dkda4p2if749h/family1/d456748798ab49d3962210861ec78fb3 into 2d76cfa810f9f3428afc40975d277f1d/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/d3449e498baa449aa7836512381a8f77_SeqId_28_ - updating store file list. 2023-11-02 03:55:36,566 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/9771d52a7de949b1b2fa1b31b5cb5e20_SeqId_28_ into 9a8949a1c626d50be08da97ed0a2c69e/family1 2023-11-02 03:55:36,566 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__ns_testMinorCompaction__bg5598r89hvsjpem36135690mie5n7cqb90bdf21lul5p00j7eehgq4ahldroche/family1/84861aa4b833475093277afa35adab69 into 9a8949a1c626d50be08da97ed0a2c69e/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/family1/9771d52a7de949b1b2fa1b31b5cb5e20_SeqId_28_) 2023-11-02 03:55:36,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__ns_testMinorCompaction__bg5598r89hvsjpem36135690mie5n7cqb90bdf21lul5p00j7eehgq4ahldroche/family1/84861aa4b833475093277afa35adab69 2023-11-02 03:55:36,567 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/d3449e498baa449aa7836512381a8f77_SeqId_28_ into 2d76cfa810f9f3428afc40975d277f1d/family1 2023-11-02 03:55:36,567 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__ns_testMinorCompaction__nb7ol65h1t8po034ij9e7rgn0d4p1njifho0uk2ka2kdrs7fe54dkda4p2if749h/family1/d456748798ab49d3962210861ec78fb3 into 2d76cfa810f9f3428afc40975d277f1d/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/family1/d3449e498baa449aa7836512381a8f77_SeqId_28_) 2023-11-02 03:55:36,567 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__ns_testMinorCompaction__nb7ol65h1t8po034ij9e7rgn0d4p1njifho0uk2ka2kdrs7fe54dkda4p2if749h/family1/d456748798ab49d3962210861ec78fb3 2023-11-02 03:55:36,568 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231102 table='ns:testMinorCompaction' column='family1' 2023-11-02 03:55:36,569 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:36,570 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231102 table='ns:testMinorCompaction' column='family1' 2023-11-02 03:55:36,570 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202311021a30e740b3584f0881edde43c58b3fb6 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202311021a30e740b3584f0881edde43c58b3fb6 2023-11-02 03:55:36,570 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:36,570 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-11-02 03:55:36,570 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20231102' 2023-11-02 03:55:36,571 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231102 for table='ns:testMinorCompaction' and column='family1' 2023-11-02 03:55:36,572 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20231102dbdca19fc46643619123e6de1fa16d3f to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20231102dbdca19fc46643619123e6de1fa16d3f 2023-11-02 03:55:36,572 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-11-02 03:55:36,572 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20231102' 2023-11-02 03:55:36,572 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20231102 for table='ns:testMinorCompaction' and column='family1' 2023-11-02 03:55:37,304 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231102e8bfa83531064a75bbb4ac154a8ff42a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20231102e8bfa83531064a75bbb4ac154a8ff42a 2023-11-02 03:55:37,305 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231102 table='ns:testMinorCompaction' column='family1' 2023-11-02 03:55:37,328 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231102/family1/19660391b2b74b4f9ebcb2092705c423 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-11-02 03:55:37,329 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231102/family1/19660391b2b74b4f9ebcb2092705c423}] 2023-11-02 03:55:37,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231102/family1/19660391b2b74b4f9ebcb2092705c423 for inclusion in 2b1e8f467362f169103ba63e90e10342/family1 2023-11-02 03:55:37,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-11-02 03:55:37,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first=2 last= 2023-11-02 03:55:37,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for 2b1e8f467362f169103ba63e90e10342: 2023-11-02 03:55:37,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231102/family1/19660391b2b74b4f9ebcb2092705c423 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__ns_testMinorCompaction__r6gfrcl5ddh28q7m7irkor8e9ptjr8kvqlbhhajdassh6tlkhutrfo0u9sjic4q/family1/19660391b2b74b4f9ebcb2092705c423 2023-11-02 03:55:37,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__ns_testMinorCompaction__r6gfrcl5ddh28q7m7irkor8e9ptjr8kvqlbhhajdassh6tlkhutrfo0u9sjic4q/family1/19660391b2b74b4f9ebcb2092705c423 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/aed10ece91c344b0850034e873de3490_SeqId_28_ 2023-11-02 03:55:37,335 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__ns_testMinorCompaction__r6gfrcl5ddh28q7m7irkor8e9ptjr8kvqlbhhajdassh6tlkhutrfo0u9sjic4q/family1/19660391b2b74b4f9ebcb2092705c423 into 2b1e8f467362f169103ba63e90e10342/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/aed10ece91c344b0850034e873de3490_SeqId_28_ - updating store file list. 2023-11-02 03:55:37,339 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/aed10ece91c344b0850034e873de3490_SeqId_28_ into 2b1e8f467362f169103ba63e90e10342/family1 2023-11-02 03:55:37,339 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__ns_testMinorCompaction__r6gfrcl5ddh28q7m7irkor8e9ptjr8kvqlbhhajdassh6tlkhutrfo0u9sjic4q/family1/19660391b2b74b4f9ebcb2092705c423 into 2b1e8f467362f169103ba63e90e10342/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/family1/aed10ece91c344b0850034e873de3490_SeqId_28_) 2023-11-02 03:55:37,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__ns_testMinorCompaction__r6gfrcl5ddh28q7m7irkor8e9ptjr8kvqlbhhajdassh6tlkhutrfo0u9sjic4q/family1/19660391b2b74b4f9ebcb2092705c423 2023-11-02 03:55:37,341 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231102 table='ns:testMinorCompaction' column='family1' 2023-11-02 03:55:37,341 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:37,343 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202311026a6980bf4f404a4d8e8ac192e65fada1 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202311026a6980bf4f404a4d8e8ac192e65fada1 2023-11-02 03:55:37,343 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-11-02 03:55:37,343 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20231102' 2023-11-02 03:55:37,343 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231102 for table='ns:testMinorCompaction' and column='family1' 2023-11-02 03:55:37,343 INFO [Listener at localhost/43647] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-11-02 03:55:37,345 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-02 03:55:37,347 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-02 03:55:37,350 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-02 03:55:37,351 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-02 03:55:37,366 INFO [Listener at localhost/43647] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=254 (was 291), OpenFileDescriptor=669 (was 796), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=105 (was 97) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=11388 (was 11007) - AvailableMemoryMB LEAK? - 2023-11-02 03:55:37,377 INFO [Listener at localhost/43647] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=254, OpenFileDescriptor=669, MaxFileDescriptor=60000, SystemLoadAverage=105, ProcessCount=167, AvailableMemoryMB=11388 2023-11-02 03:55:37,379 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-11-02 03:55:37,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:37,381 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-11-02 03:55:37,381 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-11-02 03:55:37,381 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-02 03:55:37,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-02 03:55:37,383 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39 2023-11-02 03:55:37,383 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39 empty. 2023-11-02 03:55:37,383 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39 2023-11-02 03:55:37,384 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-11-02 03:55:37,390 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-11-02 03:55:37,391 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7675): creating {ENCODED => 028924ec081a2d5ff15a43f486a54e39, NAME => 'testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp 2023-11-02 03:55:37,395 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:55:37,395 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1604): Closing 028924ec081a2d5ff15a43f486a54e39, disabling compactions & flushes 2023-11-02 03:55:37,395 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:37,395 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:37,395 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. after waiting 0 ms 2023-11-02 03:55:37,395 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:37,395 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:37,395 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1558): Region close journal for 028924ec081a2d5ff15a43f486a54e39: 2023-11-02 03:55:37,397 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-11-02 03:55:37,398 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1698897337398"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897337398"}]},"ts":"1698897337398"} 2023-11-02 03:55:37,399 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-11-02 03:55:37,400 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-02 03:55:37,400 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897337400"}]},"ts":"1698897337400"} 2023-11-02 03:55:37,401 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-11-02 03:55:37,404 INFO [PEWorker-4] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=028924ec081a2d5ff15a43f486a54e39, ASSIGN}] 2023-11-02 03:55:37,406 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=028924ec081a2d5ff15a43f486a54e39, ASSIGN 2023-11-02 03:55:37,406 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=028924ec081a2d5ff15a43f486a54e39, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=false 2023-11-02 03:55:37,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-02 03:55:37,557 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=028924ec081a2d5ff15a43f486a54e39, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:37,557 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1698897337557"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897337557"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897337557"}]},"ts":"1698897337557"} 2023-11-02 03:55:37,559 INFO [PEWorker-3] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure 028924ec081a2d5ff15a43f486a54e39, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:55:37,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-02 03:55:37,710 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:37,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:37,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 028924ec081a2d5ff15a43f486a54e39, NAME => 'testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39.', STARTKEY => '', ENDKEY => ''} 2023-11-02 03:55:37,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy 028924ec081a2d5ff15a43f486a54e39 2023-11-02 03:55:37,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:55:37,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 028924ec081a2d5ff15a43f486a54e39 2023-11-02 03:55:37,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 028924ec081a2d5ff15a43f486a54e39 2023-11-02 03:55:37,714 INFO [StoreOpener-028924ec081a2d5ff15a43f486a54e39-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 028924ec081a2d5ff15a43f486a54e39 2023-11-02 03:55:37,715 DEBUG [StoreOpener-028924ec081a2d5ff15a43f486a54e39-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1 2023-11-02 03:55:37,715 DEBUG [StoreOpener-028924ec081a2d5ff15a43f486a54e39-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1 2023-11-02 03:55:37,716 INFO [StoreOpener-028924ec081a2d5ff15a43f486a54e39-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 028924ec081a2d5ff15a43f486a54e39 columnFamilyName family1 2023-11-02 03:55:37,717 INFO [StoreOpener-028924ec081a2d5ff15a43f486a54e39-1] regionserver.HStore(310): Store=028924ec081a2d5ff15a43f486a54e39/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:55:37,717 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39 2023-11-02 03:55:37,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39 2023-11-02 03:55:37,720 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 028924ec081a2d5ff15a43f486a54e39 2023-11-02 03:55:37,721 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:55:37,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 028924ec081a2d5ff15a43f486a54e39; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62395308, jitterRate=-0.07023745775222778}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-11-02 03:55:37,721 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 028924ec081a2d5ff15a43f486a54e39: 2023-11-02 03:55:37,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39., pid=114, masterSystemTime=1698897337710 2023-11-02 03:55:37,723 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:37,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:37,723 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=028924ec081a2d5ff15a43f486a54e39, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:37,723 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1698897337723"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897337723"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897337723"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897337723"}]},"ts":"1698897337723"} 2023-11-02 03:55:37,725 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=114, resume processing ppid=113 2023-11-02 03:55:37,725 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure 028924ec081a2d5ff15a43f486a54e39, server=jenkins-hbase4.apache.org,36627,1698897201574 in 165 msec 2023-11-02 03:55:37,727 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=113, resume processing ppid=112 2023-11-02 03:55:37,727 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=028924ec081a2d5ff15a43f486a54e39, ASSIGN in 322 msec 2023-11-02 03:55:37,727 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-02 03:55:37,727 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897337727"}]},"ts":"1698897337727"} 2023-11-02 03:55:37,728 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-11-02 03:55:37,730 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-11-02 03:55:37,731 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 351 msec 2023-11-02 03:55:37,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=112 2023-11-02 03:55:37,985 INFO [Listener at localhost/43647] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-11-02 03:55:38,003 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:38,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,006 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-02 03:55:38,006 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:38,006 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-02 03:55:38,006 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-02 03:55:38,006 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,007 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,008 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:38,008 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:38,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:38,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:38,008 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:38,008 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:38,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,009 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:38,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,009 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,013 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,013 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-02 03:55:38,013 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:38,013 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-02 03:55:38,013 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:55:38,013 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:38,013 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:38,013 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. started... 2023-11-02 03:55:38,014 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 028924ec081a2d5ff15a43f486a54e39 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-11-02 03:55:38,437 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113068503529b7bb4bb9a7fb27c488b9c1b9 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113068503529b7bb4bb9a7fb27c488b9c1b9 2023-11-02 03:55:38,840 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/40c555fd7b96411f8efa3f1a24ebbc6c 2023-11-02 03:55:38,844 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/40c555fd7b96411f8efa3f1a24ebbc6c as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/40c555fd7b96411f8efa3f1a24ebbc6c 2023-11-02 03:55:38,847 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/40c555fd7b96411f8efa3f1a24ebbc6c, entries=1000, sequenceid=5, filesize=166.0 K 2023-11-02 03:55:38,848 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 028924ec081a2d5ff15a43f486a54e39 in 834ms, sequenceid=5, compaction requested=false 2023-11-02 03:55:38,848 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-11-02 03:55:38,849 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 028924ec081a2d5ff15a43f486a54e39: 2023-11-02 03:55:38,849 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:38,849 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:55:38,849 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:55:38,849 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:38,849 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-02 03:55:38,849 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-02 03:55:38,855 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:38,855 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:38,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:38,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:38,855 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,855 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:38,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:38,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:38,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:38,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:38,856 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-02 03:55:38,857 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@324e2080[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:38,857 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-02 03:55:38,857 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,858 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,858 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:38,858 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:38,858 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-02 03:55:38,858 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-02 03:55:38,858 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:38,859 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-02 03:55:38,859 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:38,859 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:38,859 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-02 03:55:38,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:38,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:38,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:38,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:38,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:38,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:38,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:38,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:38,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:38,862 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:55:38,862 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-02 03:55:38,862 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:38,862 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:38,862 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,862 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:38,864 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,865 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:38,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,865 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:38,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:38,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:38,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:38,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:38,865 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:38,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:38,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,866 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:38,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:38,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:38,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:38,868 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:38,868 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:38,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:38,869 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:38,868 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:38,869 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-02 03:55:38,869 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:38,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:38,869 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,869 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-02 03:55:38,869 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,869 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:38,869 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,869 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,869 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:38,869 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:38,869 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:38,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:38,969 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:38,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:38,974 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:38,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:55:38,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,975 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-02 03:55:38,975 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:38,975 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-02 03:55:38,975 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-02 03:55:38,976 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,976 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,977 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:38,977 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:38,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:38,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:38,977 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:38,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,978 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:38,978 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:38,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,978 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,979 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-02 03:55:38,979 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:38,979 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-02 03:55:38,979 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:38,979 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-02 03:55:38,979 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:55:38,979 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:38,979 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:38,979 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. started... 2023-11-02 03:55:38,979 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 028924ec081a2d5ff15a43f486a54e39 1/1 column families, dataSize=74 B heapSize=408 B 2023-11-02 03:55:38,993 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128f81e6f68635e485e936ec0a623e3238f to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128f81e6f68635e485e936ec0a623e3238f 2023-11-02 03:55:39,398 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/d941e5af44594b7a9bfb595eadc6ee57 2023-11-02 03:55:39,402 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/d941e5af44594b7a9bfb595eadc6ee57 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/d941e5af44594b7a9bfb595eadc6ee57 2023-11-02 03:55:39,405 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/d941e5af44594b7a9bfb595eadc6ee57, entries=1, sequenceid=9, filesize=5.0 K 2023-11-02 03:55:39,406 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 028924ec081a2d5ff15a43f486a54e39 in 427ms, sequenceid=9, compaction requested=false 2023-11-02 03:55:39,406 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 028924ec081a2d5ff15a43f486a54e39: 2023-11-02 03:55:39,406 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:39,406 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:55:39,406 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:55:39,406 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:39,406 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-02 03:55:39,406 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-02 03:55:39,408 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:39,408 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:39,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:39,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:39,408 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,408 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:39,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:39,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:39,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:39,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:39,410 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-02 03:55:39,410 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@248e6015[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:39,410 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-02 03:55:39,410 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,411 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,411 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-02 03:55:39,412 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-02 03:55:39,412 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:39,412 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:39,412 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:39,413 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-02 03:55:39,413 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:39,413 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:39,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:39,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:39,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:39,413 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-02 03:55:39,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:39,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:39,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:39,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:39,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:39,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:39,415 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:55:39,415 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-02 03:55:39,415 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:39,415 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:39,415 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,415 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:39,417 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,417 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,417 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:39,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:39,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:39,417 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:39,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:39,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:39,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,417 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:39,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:39,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:39,419 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:39,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:39,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:39,421 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:39,421 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,421 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:39,421 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:39,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:39,421 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:39,421 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-02 03:55:39,421 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:39,421 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:39,421 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:39,421 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,421 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,421 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,421 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:39,422 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:39,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:39,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:39,522 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:39,522 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:39,526 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:39,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:55:39,527 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,527 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-02 03:55:39,527 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:39,527 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-02 03:55:39,527 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-02 03:55:39,528 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,528 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,530 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:39,530 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:39,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:39,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:39,531 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:39,531 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:39,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,531 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:39,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,531 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,532 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-02 03:55:39,532 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:39,532 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-02 03:55:39,532 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:39,532 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-02 03:55:39,532 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:55:39,532 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:39,532 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:39,532 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. started... 2023-11-02 03:55:39,532 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 028924ec081a2d5ff15a43f486a54e39 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:55:39,942 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201737a0bf091704598a43b7334c0a19cb6 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201737a0bf091704598a43b7334c0a19cb6 2023-11-02 03:55:40,347 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/daea965e9783446ebd1d4762c98c1437 2023-11-02 03:55:40,351 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/daea965e9783446ebd1d4762c98c1437 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/daea965e9783446ebd1d4762c98c1437 2023-11-02 03:55:40,354 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/daea965e9783446ebd1d4762c98c1437, entries=1, sequenceid=13, filesize=4.9 K 2023-11-02 03:55:40,355 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 028924ec081a2d5ff15a43f486a54e39 in 823ms, sequenceid=13, compaction requested=true 2023-11-02 03:55:40,355 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 028924ec081a2d5ff15a43f486a54e39: 2023-11-02 03:55:40,355 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:40,355 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:55:40,355 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:55:40,355 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:40,355 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-02 03:55:40,355 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-02 03:55:40,357 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,357 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:40,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:40,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:40,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:40,357 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,357 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:40,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:40,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:40,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:40,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:40,359 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-02 03:55:40,359 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@32c73f3f[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:40,359 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-02 03:55:40,359 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,360 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,360 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-02 03:55:40,360 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-02 03:55:40,360 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:40,360 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:40,360 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:40,362 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:40,362 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-02 03:55:40,362 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:40,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:40,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:40,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:40,362 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-02 03:55:40,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:40,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:40,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:40,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:40,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:40,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:40,364 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:55:40,364 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-02 03:55:40,364 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:40,364 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:40,364 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,364 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:40,367 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,367 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:40,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,367 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:40,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:40,367 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:40,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:40,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:40,369 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:40,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:40,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:40,374 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:40,374 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:40,374 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-02 03:55:40,374 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:40,374 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:40,374 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:40,374 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:40,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:40,374 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:40,374 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:40,374 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,374 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:40,374 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,374 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,374 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,374 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:40,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:40,474 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:40,475 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:40,478 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:40,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:55:40,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,480 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-02 03:55:40,480 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:40,480 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-02 03:55:40,480 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-02 03:55:40,481 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,481 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,482 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:40,482 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:40,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:40,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:40,482 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:40,482 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:40,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,483 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:40,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,483 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,483 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-02 03:55:40,483 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:40,483 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-02 03:55:40,483 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:40,483 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-02 03:55:40,484 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:55:40,484 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:40,484 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:40,484 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. started... 2023-11-02 03:55:40,484 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 028924ec081a2d5ff15a43f486a54e39 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:55:40,895 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512059e23b866fc5f4dd6addf9465459855ef to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512059e23b866fc5f4dd6addf9465459855ef 2023-11-02 03:55:41,300 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/3ea133d26899430fa3402ecbd1617e43 2023-11-02 03:55:41,304 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/3ea133d26899430fa3402ecbd1617e43 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/3ea133d26899430fa3402ecbd1617e43 2023-11-02 03:55:41,307 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/3ea133d26899430fa3402ecbd1617e43, entries=1, sequenceid=17, filesize=4.9 K 2023-11-02 03:55:41,308 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 028924ec081a2d5ff15a43f486a54e39 in 824ms, sequenceid=17, compaction requested=true 2023-11-02 03:55:41,308 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 028924ec081a2d5ff15a43f486a54e39: 2023-11-02 03:55:41,308 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:41,308 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:55:41,308 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:55:41,308 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:41,309 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-02 03:55:41,309 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-02 03:55:41,311 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:41,311 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:41,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:41,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:41,311 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,311 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:41,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:41,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:41,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:41,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:41,313 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-02 03:55:41,313 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b827996[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:41,313 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-02 03:55:41,313 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,314 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,314 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-02 03:55:41,314 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-02 03:55:41,314 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:41,314 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:41,314 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:41,315 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-02 03:55:41,315 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:41,315 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:41,315 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-02 03:55:41,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:41,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:41,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:41,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:41,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:41,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:41,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:41,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:41,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:41,318 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:55:41,318 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-02 03:55:41,318 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:41,318 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:41,318 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,318 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:41,319 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,319 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,319 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:41,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:41,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:41,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:41,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:41,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:41,320 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:41,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:41,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,321 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:41,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:41,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:41,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:41,325 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:41,325 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,325 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:41,325 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:41,325 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-02 03:55:41,325 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:41,325 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:41,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:41,325 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:41,325 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-02 03:55:41,325 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:41,325 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,325 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,325 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:41,325 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:41,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:41,325 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:41,426 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:41,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:41,430 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:41,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:55:41,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,431 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-02 03:55:41,431 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:41,431 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-02 03:55:41,431 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-02 03:55:41,432 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,432 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,433 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:41,433 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:41,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:41,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:41,434 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:41,434 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:41,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,434 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:41,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,434 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,435 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-02 03:55:41,435 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:41,435 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-02 03:55:41,435 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:41,435 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-02 03:55:41,435 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:55:41,435 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:41,435 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:41,435 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. started... 2023-11-02 03:55:41,435 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 028924ec081a2d5ff15a43f486a54e39 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:55:41,846 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512285ec57679899d400f81ac6e0ff81dfe6b to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512285ec57679899d400f81ac6e0ff81dfe6b 2023-11-02 03:55:42,251 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/7982659cc7d7463695e6f91bf5c3adfb 2023-11-02 03:55:42,255 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/7982659cc7d7463695e6f91bf5c3adfb as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/7982659cc7d7463695e6f91bf5c3adfb 2023-11-02 03:55:42,258 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/7982659cc7d7463695e6f91bf5c3adfb, entries=1, sequenceid=21, filesize=4.9 K 2023-11-02 03:55:42,258 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 028924ec081a2d5ff15a43f486a54e39 in 823ms, sequenceid=21, compaction requested=true 2023-11-02 03:55:42,259 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 028924ec081a2d5ff15a43f486a54e39: 2023-11-02 03:55:42,259 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:42,259 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:55:42,259 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:55:42,259 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:42,259 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-02 03:55:42,259 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-02 03:55:42,261 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:42,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:42,261 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:42,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:42,261 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,261 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:42,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:42,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:42,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:42,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:42,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-02 03:55:42,262 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@488b1246[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:42,262 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-02 03:55:42,262 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,264 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,264 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-02 03:55:42,264 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-02 03:55:42,264 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:42,264 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:42,264 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:42,266 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-02 03:55:42,266 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:42,266 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:42,266 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-02 03:55:42,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:42,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:42,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:42,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:42,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:42,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:42,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:42,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:42,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:42,268 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:55:42,268 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-02 03:55:42,268 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:42,268 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:42,269 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,269 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:42,270 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,270 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,270 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:42,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:42,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:42,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:42,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:42,270 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:42,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:42,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:42,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,271 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:42,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:42,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:42,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:42,274 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:42,274 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,274 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,274 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:42,274 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:42,274 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-02 03:55:42,274 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:42,274 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:42,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:42,274 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:42,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:42,274 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:42,274 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,274 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:42,274 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-02 03:55:42,274 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:42,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:42,274 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,374 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:42,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:42,379 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:42,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:55:42,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,380 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-02 03:55:42,380 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:42,380 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-02 03:55:42,380 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-02 03:55:42,381 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,381 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,382 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:42,382 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:42,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:42,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:42,382 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:42,382 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:42,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,382 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:42,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,383 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,383 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-02 03:55:42,383 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:42,383 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-02 03:55:42,383 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:42,383 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-02 03:55:42,383 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:55:42,383 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:42,383 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:42,383 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. started... 2023-11-02 03:55:42,384 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 028924ec081a2d5ff15a43f486a54e39 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:55:42,794 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512316365a3c214374c6aa54a52a9ff6196e5 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512316365a3c214374c6aa54a52a9ff6196e5 2023-11-02 03:55:43,199 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/c3cf179eeeaf430d91398c4605e24480 2023-11-02 03:55:43,203 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/c3cf179eeeaf430d91398c4605e24480 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/c3cf179eeeaf430d91398c4605e24480 2023-11-02 03:55:43,206 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/c3cf179eeeaf430d91398c4605e24480, entries=1, sequenceid=25, filesize=4.9 K 2023-11-02 03:55:43,206 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 028924ec081a2d5ff15a43f486a54e39 in 822ms, sequenceid=25, compaction requested=true 2023-11-02 03:55:43,206 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 028924ec081a2d5ff15a43f486a54e39: 2023-11-02 03:55:43,206 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:43,206 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:55:43,206 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:55:43,206 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:43,207 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-02 03:55:43,207 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-02 03:55:43,209 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:43,209 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:43,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:43,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:43,209 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,209 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:43,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:43,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:43,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:43,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:43,211 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-02 03:55:43,211 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1fd25d8a[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:43,211 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-02 03:55:43,211 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,212 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,212 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-02 03:55:43,212 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-02 03:55:43,212 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:43,212 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:43,212 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:43,213 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-02 03:55:43,213 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:43,213 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:43,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:43,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:43,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:43,213 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-02 03:55:43,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:43,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:43,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:43,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:43,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:43,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:43,216 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:55:43,216 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-02 03:55:43,216 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:43,216 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:43,216 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,216 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:43,220 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,220 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:43,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,220 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:43,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:43,221 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:43,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:43,221 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:43,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:43,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:43,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,222 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:43,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:43,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:43,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:43,224 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:43,224 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,224 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,224 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:43,224 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:43,224 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:43,224 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:43,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:43,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:43,224 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-02 03:55:43,224 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-02 03:55:43,225 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:43,225 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,225 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,225 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:43,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:43,225 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:43,225 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:43,253 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-02 03:55:43,325 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:43,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:43,329 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:43,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:55:43,331 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,331 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-02 03:55:43,331 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:43,331 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-02 03:55:43,331 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-02 03:55:43,331 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,331 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,333 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:43,333 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:43,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:43,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:43,333 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:43,333 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:43,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,333 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:43,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,334 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,334 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-02 03:55:43,334 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:43,334 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-02 03:55:43,334 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:43,334 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-02 03:55:43,334 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:55:43,334 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:43,334 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:43,334 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. started... 2023-11-02 03:55:43,335 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 028924ec081a2d5ff15a43f486a54e39 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:55:43,745 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010105d07babf8c44d4abce8c08932524f7a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010105d07babf8c44d4abce8c08932524f7a 2023-11-02 03:55:44,150 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/814191508ac24f35a93d95a5dea97499 2023-11-02 03:55:44,154 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/814191508ac24f35a93d95a5dea97499 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/814191508ac24f35a93d95a5dea97499 2023-11-02 03:55:44,157 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/814191508ac24f35a93d95a5dea97499, entries=1, sequenceid=29, filesize=4.9 K 2023-11-02 03:55:44,158 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 028924ec081a2d5ff15a43f486a54e39 in 823ms, sequenceid=29, compaction requested=true 2023-11-02 03:55:44,158 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 028924ec081a2d5ff15a43f486a54e39: 2023-11-02 03:55:44,158 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:44,158 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:55:44,158 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:55:44,158 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:44,158 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-02 03:55:44,158 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-02 03:55:44,160 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:44,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:44,160 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:44,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:44,160 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,160 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:44,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:44,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:44,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:44,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:44,162 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-02 03:55:44,162 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1467b788[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:44,162 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-02 03:55:44,162 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,164 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,164 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-02 03:55:44,164 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-02 03:55:44,164 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:44,164 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:44,164 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:44,165 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:44,165 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-02 03:55:44,165 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:44,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:44,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:44,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:44,165 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-02 03:55:44,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:44,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:44,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:44,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:44,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:44,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:44,167 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:55:44,167 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-02 03:55:44,168 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:44,168 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:44,168 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,168 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:44,169 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,169 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:44,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:44,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,169 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:44,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:44,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:44,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:44,170 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:44,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:44,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:44,171 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:44,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:44,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:44,173 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:44,173 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:44,173 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:44,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:44,173 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-02 03:55:44,173 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:44,173 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:44,173 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:44,173 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,173 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:44,173 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:44,173 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:44,173 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,174 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:44,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:44,174 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,274 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:44,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:44,278 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:44,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:55:44,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,279 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-02 03:55:44,279 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:44,279 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-02 03:55:44,279 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-02 03:55:44,280 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,280 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,283 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:44,283 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:44,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:44,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:44,283 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:44,283 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:44,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,283 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:44,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,284 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,284 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-02 03:55:44,284 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:44,284 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-02 03:55:44,284 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:44,284 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-02 03:55:44,284 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:55:44,284 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:44,284 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:44,284 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. started... 2023-11-02 03:55:44,284 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 028924ec081a2d5ff15a43f486a54e39 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:55:44,695 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103112675d15a444502bb2cde64c92eecf9 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103112675d15a444502bb2cde64c92eecf9 2023-11-02 03:55:45,099 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/ed092100f1f245a4aa7ee696410b6313 2023-11-02 03:55:45,103 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/ed092100f1f245a4aa7ee696410b6313 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/ed092100f1f245a4aa7ee696410b6313 2023-11-02 03:55:45,106 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/ed092100f1f245a4aa7ee696410b6313, entries=1, sequenceid=33, filesize=4.9 K 2023-11-02 03:55:45,107 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 028924ec081a2d5ff15a43f486a54e39 in 823ms, sequenceid=33, compaction requested=true 2023-11-02 03:55:45,107 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 028924ec081a2d5ff15a43f486a54e39: 2023-11-02 03:55:45,107 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:45,107 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:55:45,107 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:55:45,107 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:45,107 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-02 03:55:45,107 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-02 03:55:45,109 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:45,109 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:45,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:45,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:45,109 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,109 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:45,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:45,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:45,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:45,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:45,111 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-02 03:55:45,111 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-02 03:55:45,111 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,111 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c31325b[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:45,112 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,112 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,112 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-02 03:55:45,112 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-02 03:55:45,112 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:45,112 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:45,112 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:45,113 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:45,113 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-02 03:55:45,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:45,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:45,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:45,113 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:45,113 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-02 03:55:45,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:45,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:45,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:45,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:45,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:45,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:45,116 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:55:45,116 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-02 03:55:45,116 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:45,116 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:45,116 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,116 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:45,122 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,122 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:45,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:45,122 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:45,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:45,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:45,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:45,123 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:45,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:45,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:45,124 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:45,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:45,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:45,126 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:45,126 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:45,126 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,126 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:45,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:45,126 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:45,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-02 03:55:45,126 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-02 03:55:45,127 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:45,126 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:45,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:45,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:45,126 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:45,127 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,127 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,127 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,127 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:45,127 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:45,227 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:45,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:45,231 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:45,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:55:45,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,233 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-02 03:55:45,233 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:45,233 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-02 03:55:45,233 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-02 03:55:45,234 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,234 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,235 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:45,235 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:45,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:45,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:45,235 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:45,235 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:45,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,236 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:45,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,236 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,236 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-02 03:55:45,236 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:45,236 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-02 03:55:45,236 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:45,236 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-02 03:55:45,236 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:55:45,237 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:45,237 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:45,237 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. started... 2023-11-02 03:55:45,237 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 028924ec081a2d5ff15a43f486a54e39 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:55:45,647 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509071958ec48ce1449baa07ebe88e61363e3 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201509071958ec48ce1449baa07ebe88e61363e3 2023-11-02 03:55:46,052 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/6103ec26b483442c8a28779fc2a9967a 2023-11-02 03:55:46,056 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/6103ec26b483442c8a28779fc2a9967a as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/6103ec26b483442c8a28779fc2a9967a 2023-11-02 03:55:46,059 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/6103ec26b483442c8a28779fc2a9967a, entries=1, sequenceid=37, filesize=4.9 K 2023-11-02 03:55:46,059 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 028924ec081a2d5ff15a43f486a54e39 in 822ms, sequenceid=37, compaction requested=true 2023-11-02 03:55:46,059 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 028924ec081a2d5ff15a43f486a54e39: 2023-11-02 03:55:46,059 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:46,060 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:55:46,060 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:55:46,060 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:46,060 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-02 03:55:46,060 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-02 03:55:46,066 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:46,066 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:46,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:46,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:46,066 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,066 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:46,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:46,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:46,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:46,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:46,068 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-02 03:55:46,068 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1c3808b9[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:46,068 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-02 03:55:46,068 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,069 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,069 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-02 03:55:46,069 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:46,069 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:46,069 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-02 03:55:46,069 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:46,070 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:46,070 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-02 03:55:46,070 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:46,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:46,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:46,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:46,071 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-02 03:55:46,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:46,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:46,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:46,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:46,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:46,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:46,073 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:55:46,073 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-02 03:55:46,073 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:46,073 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:46,073 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,073 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:46,074 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,074 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,074 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:46,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:46,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:46,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:46,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:46,075 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:46,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:46,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:46,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,076 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:46,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:46,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:46,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:46,080 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:46,080 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:46,080 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:46,080 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-02 03:55:46,080 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-02 03:55:46,080 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:46,080 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:46,080 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,080 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:46,080 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:46,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:46,080 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,080 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,080 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,080 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:46,080 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:46,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:46,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:46,181 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:46,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:46,185 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:46,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-11-02 03:55:46,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,187 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-11-02 03:55:46,187 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:46,187 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-11-02 03:55:46,187 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-11-02 03:55:46,187 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,187 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,189 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:46,189 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:46,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:46,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:46,189 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:46,189 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:46,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,189 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:46,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,190 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,190 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-11-02 03:55:46,190 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:46,190 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-11-02 03:55:46,190 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:46,190 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-11-02 03:55:46,190 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-11-02 03:55:46,190 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:46,190 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:46,190 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. started... 2023-11-02 03:55:46,190 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 028924ec081a2d5ff15a43f486a54e39 1/1 column families, dataSize=73 B heapSize=408 B 2023-11-02 03:55:46,600 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511209683a3ff464a4002921ee1213d96db90 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511209683a3ff464a4002921ee1213d96db90 2023-11-02 03:55:47,005 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/a0faf614114d46318efa041845910d08 2023-11-02 03:55:47,009 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/a0faf614114d46318efa041845910d08 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/a0faf614114d46318efa041845910d08 2023-11-02 03:55:47,012 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/a0faf614114d46318efa041845910d08, entries=1, sequenceid=41, filesize=4.9 K 2023-11-02 03:55:47,013 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 028924ec081a2d5ff15a43f486a54e39 in 823ms, sequenceid=41, compaction requested=true 2023-11-02 03:55:47,013 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 028924ec081a2d5ff15a43f486a54e39: 2023-11-02 03:55:47,013 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:47,013 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-11-02 03:55:47,013 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-11-02 03:55:47,013 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:47,013 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-11-02 03:55:47,013 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-11-02 03:55:47,015 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:47,015 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:47,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:47,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:47,015 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,015 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:47,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:47,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:47,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:47,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:47,017 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-11-02 03:55:47,017 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@110b38b9[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:47,017 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-11-02 03:55:47,017 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,018 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,018 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-11-02 03:55:47,018 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-11-02 03:55:47,018 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:47,018 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:47,018 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:47,020 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:47,021 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-11-02 03:55:47,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:47,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:47,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:47,021 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:47,021 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-11-02 03:55:47,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:47,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:47,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:47,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:47,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:47,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:47,023 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-11-02 03:55:47,023 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-11-02 03:55:47,023 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:47,023 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:47,023 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,023 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:47,024 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,024 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:47,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:47,024 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:47,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,025 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:47,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:47,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:47,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:47,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:47,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,026 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:47,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:47,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:47,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:47,028 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:47,028 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,028 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:47,028 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:47,028 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:47,028 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,028 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,028 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,028 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-11-02 03:55:47,029 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-11-02 03:55:47,028 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:47,028 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:47,029 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:47,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:47,029 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:47,029 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:47,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:47,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:47,129 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:47,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:47,131 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41275] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-11-02 03:55:47,133 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] procedure2.ProcedureExecutor(1028): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-02 03:55:47,133 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-02 03:55:47,134 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698897947134 2023-11-02 03:55:47,134 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:55:47,135 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-11-02 03:55:47,135 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-11-02 03:55:47,135 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-02 03:55:47,136 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] zookeeper.ReadOnlyZKClient(139): Connect 0x749c8902 to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:55:47,140 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@297eb670, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:55:47,140 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-02 03:55:47,563 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103367bb72e552c44be987132c3e230ffa4 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103367bb72e552c44be987132c3e230ffa4 2023-11-02 03:55:47,563 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-02 03:55:47,576 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-02 03:55:47,578 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55988, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-02 03:55:47,583 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-02 03:55:47,584 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42802, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-02 03:55:47,592 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/ed654150a8f14206af8febfde197d46d first=Optional[r5] last=Optional[r6] 2023-11-02 03:55:47,592 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/ed654150a8f14206af8febfde197d46d}] 2023-11-02 03:55:47,598 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/ed654150a8f14206af8febfde197d46d for inclusion in 028924ec081a2d5ff15a43f486a54e39/family1 2023-11-02 03:55:47,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-11-02 03:55:47,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last= 2023-11-02 03:55:47,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for 028924ec081a2d5ff15a43f486a54e39: 2023-11-02 03:55:47,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/ed654150a8f14206af8febfde197d46d to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__o89q67rovufmsmkqiikfssi6tv3ih3j7m2jui6a7p3ql2ncaufh9nhd3ivf72eg4/family1/ed654150a8f14206af8febfde197d46d 2023-11-02 03:55:47,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__o89q67rovufmsmkqiikfssi6tv3ih3j7m2jui6a7p3ql2ncaufh9nhd3ivf72eg4/family1/ed654150a8f14206af8febfde197d46d as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/bf111f05a57f42ad8522fa0a3b5c7b8a_SeqId_44_ 2023-11-02 03:55:47,604 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__o89q67rovufmsmkqiikfssi6tv3ih3j7m2jui6a7p3ql2ncaufh9nhd3ivf72eg4/family1/ed654150a8f14206af8febfde197d46d into 028924ec081a2d5ff15a43f486a54e39/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/bf111f05a57f42ad8522fa0a3b5c7b8a_SeqId_44_ - updating store file list. 2023-11-02 03:55:47,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/bf111f05a57f42ad8522fa0a3b5c7b8a_SeqId_44_ into 028924ec081a2d5ff15a43f486a54e39/family1 2023-11-02 03:55:47,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__o89q67rovufmsmkqiikfssi6tv3ih3j7m2jui6a7p3ql2ncaufh9nhd3ivf72eg4/family1/ed654150a8f14206af8febfde197d46d into 028924ec081a2d5ff15a43f486a54e39/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/bf111f05a57f42ad8522fa0a3b5c7b8a_SeqId_44_) 2023-11-02 03:55:47,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__o89q67rovufmsmkqiikfssi6tv3ih3j7m2jui6a7p3ql2ncaufh9nhd3ivf72eg4/family1/ed654150a8f14206af8febfde197d46d 2023-11-02 03:55:47,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-11-02 03:55:47,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-11-02 03:55:47,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(1912): 028924ec081a2d5ff15a43f486a54e39/family1 is initiating minor compaction 2023-11-02 03:55:47,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1b8855e7; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:55:47,610 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 028924ec081a2d5ff15a43f486a54e39/family1 in testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:47,610 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/d941e5af44594b7a9bfb595eadc6ee57, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/daea965e9783446ebd1d4762c98c1437, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/3ea133d26899430fa3402ecbd1617e43, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/7982659cc7d7463695e6f91bf5c3adfb, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/c3cf179eeeaf430d91398c4605e24480, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/814191508ac24f35a93d95a5dea97499, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/ed092100f1f245a4aa7ee696410b6313, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/6103ec26b483442c8a28779fc2a9967a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/a0faf614114d46318efa041845910d08, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/bf111f05a57f42ad8522fa0a3b5c7b8a_SeqId_44_] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp, totalSize=49.7 K 2023-11-02 03:55:47,610 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting d941e5af44594b7a9bfb595eadc6ee57, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-11-02 03:55:47,611 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting daea965e9783446ebd1d4762c98c1437, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-11-02 03:55:47,611 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 3ea133d26899430fa3402ecbd1617e43, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-11-02 03:55:47,611 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 7982659cc7d7463695e6f91bf5c3adfb, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-11-02 03:55:47,611 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting c3cf179eeeaf430d91398c4605e24480, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-11-02 03:55:47,611 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 814191508ac24f35a93d95a5dea97499, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-11-02 03:55:47,611 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting ed092100f1f245a4aa7ee696410b6313, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-11-02 03:55:47,611 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 6103ec26b483442c8a28779fc2a9967a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-11-02 03:55:47,611 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting a0faf614114d46318efa041845910d08, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-11-02 03:55:47,611 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting bf111f05a57f42ad8522fa0a3b5c7b8a_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-11-02 03:55:47,613 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-02 03:55:47,614 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:47,616 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010105d07babf8c44d4abce8c08932524f7a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2016010105d07babf8c44d4abce8c08932524f7a 2023-11-02 03:55:47,617 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103112675d15a444502bb2cde64c92eecf9 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103112675d15a444502bb2cde64c92eecf9 2023-11-02 03:55:47,617 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-11-02 03:55:47,617 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-11-02 03:55:47,617 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-02 03:55:47,617 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-02 03:55:47,641 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] throttle.PressureAwareThroughputController(145): 028924ec081a2d5ff15a43f486a54e39#family1#compaction#165 average throughput is 1.37 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-11-02 03:55:47,657 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/f79305e83abc4e0cb40c2614be8fa323 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/f79305e83abc4e0cb40c2614be8fa323 2023-11-02 03:55:47,661 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 028924ec081a2d5ff15a43f486a54e39/family1 of 028924ec081a2d5ff15a43f486a54e39 into f79305e83abc4e0cb40c2614be8fa323(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-02 03:55:47,661 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 028924ec081a2d5ff15a43f486a54e39: 2023-11-02 03:55:47,661 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1698897347610; duration=0sec 2023-11-02 03:55:47,662 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:55:47,706 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/d941e5af44594b7a9bfb595eadc6ee57, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/daea965e9783446ebd1d4762c98c1437, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/3ea133d26899430fa3402ecbd1617e43, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/7982659cc7d7463695e6f91bf5c3adfb, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/c3cf179eeeaf430d91398c4605e24480, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/814191508ac24f35a93d95a5dea97499, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/ed092100f1f245a4aa7ee696410b6313, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/6103ec26b483442c8a28779fc2a9967a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/a0faf614114d46318efa041845910d08, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/bf111f05a57f42ad8522fa0a3b5c7b8a_SeqId_44_] to archive 2023-11-02 03:55:47,706 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:47,708 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/d941e5af44594b7a9bfb595eadc6ee57 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/d941e5af44594b7a9bfb595eadc6ee57 2023-11-02 03:55:47,709 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/daea965e9783446ebd1d4762c98c1437 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/daea965e9783446ebd1d4762c98c1437 2023-11-02 03:55:47,710 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/3ea133d26899430fa3402ecbd1617e43 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/3ea133d26899430fa3402ecbd1617e43 2023-11-02 03:55:47,711 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/7982659cc7d7463695e6f91bf5c3adfb to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/7982659cc7d7463695e6f91bf5c3adfb 2023-11-02 03:55:47,712 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/c3cf179eeeaf430d91398c4605e24480 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/c3cf179eeeaf430d91398c4605e24480 2023-11-02 03:55:47,713 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/814191508ac24f35a93d95a5dea97499 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/814191508ac24f35a93d95a5dea97499 2023-11-02 03:55:47,714 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/ed092100f1f245a4aa7ee696410b6313 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/ed092100f1f245a4aa7ee696410b6313 2023-11-02 03:55:47,715 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/6103ec26b483442c8a28779fc2a9967a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/6103ec26b483442c8a28779fc2a9967a 2023-11-02 03:55:47,716 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/a0faf614114d46318efa041845910d08 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/a0faf614114d46318efa041845910d08 2023-11-02 03:55:47,717 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/bf111f05a57f42ad8522fa0a3b5c7b8a_SeqId_44_ to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/bf111f05a57f42ad8522fa0a3b5c7b8a_SeqId_44_ 2023-11-02 03:55:48,062 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231a519d673a8234e22a4863543edc48195 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231a519d673a8234e22a4863543edc48195 2023-11-02 03:55:48,063 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-02 03:55:48,086 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/9d4c78c4556b4e729ad813783cf4a529 first=Optional[r1] last=Optional[r4] 2023-11-02 03:55:48,087 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/9d4c78c4556b4e729ad813783cf4a529}] 2023-11-02 03:55:48,089 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/9d4c78c4556b4e729ad813783cf4a529 for inclusion in 028924ec081a2d5ff15a43f486a54e39/family1 2023-11-02 03:55:48,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-11-02 03:55:48,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last= 2023-11-02 03:55:48,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for 028924ec081a2d5ff15a43f486a54e39: 2023-11-02 03:55:48,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/9d4c78c4556b4e729ad813783cf4a529 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__igi22cgpk6besac90orveoeo2et0q8r4aug6bmcn7dknk7a8lau6p1co6956l9f1/family1/9d4c78c4556b4e729ad813783cf4a529 2023-11-02 03:55:48,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__igi22cgpk6besac90orveoeo2et0q8r4aug6bmcn7dknk7a8lau6p1co6956l9f1/family1/9d4c78c4556b4e729ad813783cf4a529 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/72e5ac87db5a4f0ab9e9dc73c223eb64_SeqId_47_ 2023-11-02 03:55:48,094 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__igi22cgpk6besac90orveoeo2et0q8r4aug6bmcn7dknk7a8lau6p1co6956l9f1/family1/9d4c78c4556b4e729ad813783cf4a529 into 028924ec081a2d5ff15a43f486a54e39/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/72e5ac87db5a4f0ab9e9dc73c223eb64_SeqId_47_ - updating store file list. 2023-11-02 03:55:48,098 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/72e5ac87db5a4f0ab9e9dc73c223eb64_SeqId_47_ into 028924ec081a2d5ff15a43f486a54e39/family1 2023-11-02 03:55:48,098 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__igi22cgpk6besac90orveoeo2et0q8r4aug6bmcn7dknk7a8lau6p1co6956l9f1/family1/9d4c78c4556b4e729ad813783cf4a529 into 028924ec081a2d5ff15a43f486a54e39/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/72e5ac87db5a4f0ab9e9dc73c223eb64_SeqId_47_) 2023-11-02 03:55:48,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__igi22cgpk6besac90orveoeo2et0q8r4aug6bmcn7dknk7a8lau6p1co6956l9f1/family1/9d4c78c4556b4e729ad813783cf4a529 2023-11-02 03:55:48,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-11-02 03:55:48,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-11-02 03:55:48,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(1912): 028924ec081a2d5ff15a43f486a54e39/family1 is initiating minor compaction (all files) 2023-11-02 03:55:48,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6afa715b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:55:48,099 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 028924ec081a2d5ff15a43f486a54e39/family1 in testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:55:48,099 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/40c555fd7b96411f8efa3f1a24ebbc6c, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/f79305e83abc4e0cb40c2614be8fa323, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/72e5ac87db5a4f0ab9e9dc73c223eb64_SeqId_47_] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp, totalSize=178.1 K 2023-11-02 03:55:48,100 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 40c555fd7b96411f8efa3f1a24ebbc6c, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878824741 2023-11-02 03:55:48,100 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting f79305e83abc4e0cb40c2614be8fa323, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621224741 2023-11-02 03:55:48,101 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] compactions.Compactor(207): Compacting 72e5ac87db5a4f0ab9e9dc73c223eb64_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965224741 2023-11-02 03:55:48,101 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-02 03:55:48,101 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:48,103 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201737a0bf091704598a43b7334c0a19cb6 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151201737a0bf091704598a43b7334c0a19cb6 2023-11-02 03:55:48,104 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512059e23b866fc5f4dd6addf9465459855ef to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512059e23b866fc5f4dd6addf9465459855ef 2023-11-02 03:55:48,105 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512285ec57679899d400f81ac6e0ff81dfe6b to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512285ec57679899d400f81ac6e0ff81dfe6b 2023-11-02 03:55:48,106 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512316365a3c214374c6aa54a52a9ff6196e5 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512316365a3c214374c6aa54a52a9ff6196e5 2023-11-02 03:55:48,106 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-11-02 03:55:48,106 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-11-02 03:55:48,106 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-02 03:55:48,106 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-02 03:55:48,113 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] throttle.PressureAwareThroughputController(145): 028924ec081a2d5ff15a43f486a54e39#family1#compaction#166 average throughput is 25.93 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 57.69 MB/second 2023-11-02 03:55:48,145 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/.tmp/family1/205470ae359944c4a6029d98add34a92 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/205470ae359944c4a6029d98add34a92 2023-11-02 03:55:48,150 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 028924ec081a2d5ff15a43f486a54e39/family1 of 028924ec081a2d5ff15a43f486a54e39 into 205470ae359944c4a6029d98add34a92(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-02 03:55:48,150 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 028924ec081a2d5ff15a43f486a54e39: 2023-11-02 03:55:48,150 INFO [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1698897348099; duration=0sec 2023-11-02 03:55:48,150 DEBUG [RS:0;jenkins-hbase4:36627-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-11-02 03:55:48,206 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/40c555fd7b96411f8efa3f1a24ebbc6c, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/f79305e83abc4e0cb40c2614be8fa323, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/72e5ac87db5a4f0ab9e9dc73c223eb64_SeqId_47_] to archive 2023-11-02 03:55:48,206 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:48,208 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/40c555fd7b96411f8efa3f1a24ebbc6c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/40c555fd7b96411f8efa3f1a24ebbc6c 2023-11-02 03:55:48,209 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/f79305e83abc4e0cb40c2614be8fa323 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/f79305e83abc4e0cb40c2614be8fa323 2023-11-02 03:55:48,210 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/72e5ac87db5a4f0ab9e9dc73c223eb64_SeqId_47_ to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/72e5ac87db5a4f0ab9e9dc73c223eb64_SeqId_47_ 2023-11-02 03:55:48,549 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511308fe861a288b64f19a74a78d0390ed13f to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511308fe861a288b64f19a74a78d0390ed13f 2023-11-02 03:55:48,551 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-02 03:55:48,577 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/27911a5c572d4249bcdd183eafda4f2c first=Optional[r00] last=Optional[r8] 2023-11-02 03:55:48,578 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/27911a5c572d4249bcdd183eafda4f2c}] 2023-11-02 03:55:48,580 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/27911a5c572d4249bcdd183eafda4f2c for inclusion in 028924ec081a2d5ff15a43f486a54e39/family1 2023-11-02 03:55:48,582 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-11-02 03:55:48,582 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last= 2023-11-02 03:55:48,582 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for 028924ec081a2d5ff15a43f486a54e39: 2023-11-02 03:55:48,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/27911a5c572d4249bcdd183eafda4f2c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__j12glgiolmh6j2ggqdosaj4vq7bhiunliuosle5sdr7ln8fovmfckundog4bifs1/family1/27911a5c572d4249bcdd183eafda4f2c 2023-11-02 03:55:48,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__j12glgiolmh6j2ggqdosaj4vq7bhiunliuosle5sdr7ln8fovmfckundog4bifs1/family1/27911a5c572d4249bcdd183eafda4f2c as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/3353f4206435465a999486b9c7f88e45_SeqId_50_ 2023-11-02 03:55:48,585 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__j12glgiolmh6j2ggqdosaj4vq7bhiunliuosle5sdr7ln8fovmfckundog4bifs1/family1/27911a5c572d4249bcdd183eafda4f2c into 028924ec081a2d5ff15a43f486a54e39/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/3353f4206435465a999486b9c7f88e45_SeqId_50_ - updating store file list. 2023-11-02 03:55:48,588 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/3353f4206435465a999486b9c7f88e45_SeqId_50_ into 028924ec081a2d5ff15a43f486a54e39/family1 2023-11-02 03:55:48,588 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__j12glgiolmh6j2ggqdosaj4vq7bhiunliuosle5sdr7ln8fovmfckundog4bifs1/family1/27911a5c572d4249bcdd183eafda4f2c into 028924ec081a2d5ff15a43f486a54e39/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/family1/3353f4206435465a999486b9c7f88e45_SeqId_50_) 2023-11-02 03:55:48,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMinorCompactionWithMonthlyPolicy__j12glgiolmh6j2ggqdosaj4vq7bhiunliuosle5sdr7ln8fovmfckundog4bifs1/family1/27911a5c572d4249bcdd183eafda4f2c 2023-11-02 03:55:48,590 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-02 03:55:48,590 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:55:48,593 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511209683a3ff464a4002921ee1213d96db90 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511209683a3ff464a4002921ee1213d96db90 2023-11-02 03:55:48,594 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128f81e6f68635e485e936ec0a623e3238f to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128f81e6f68635e485e936ec0a623e3238f 2023-11-02 03:55:48,595 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113068503529b7bb4bb9a7fb27c488b9c1b9 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015113068503529b7bb4bb9a7fb27c488b9c1b9 2023-11-02 03:55:48,595 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-11-02 03:55:48,595 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-11-02 03:55:48,596 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-02 03:55:48,596 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-02 03:55:48,596 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-11-02 03:55:48,596 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-11-02 03:55:48,596 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:55:48,597 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-11-02 03:55:48,598 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.4650 sec 2023-11-02 03:55:48,789 INFO [Listener at localhost/43647] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=263 (was 254) - Thread LEAK? -, OpenFileDescriptor=686 (was 669) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=112 (was 105) - SystemLoadAverage LEAK? -, ProcessCount=167 (was 167), AvailableMemoryMB=11476 (was 11388) - AvailableMemoryMB LEAK? - 2023-11-02 03:55:48,803 INFO [Listener at localhost/43647] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=263, OpenFileDescriptor=686, MaxFileDescriptor=60000, SystemLoadAverage=112, ProcessCount=167, AvailableMemoryMB=11476 2023-11-02 03:55:48,822 DEBUG [Listener at localhost/43647] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-11-02 03:55:48,875 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10R\xBC\xA9\x93m4\x9FG"\x5CJXyR:\x87"\x10\xCB\xA0\xFE\x99IM\x0E\x9C\x13\xD73{\xAA\x9C\x1Ev*\x10\xF1;\x92#Mx\xFD\x0CnK\x85\x0C#\x80\xC6\xDE2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-11-02 03:55:48,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure2.ProcedureExecutor(1028): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-11-02 03:55:48,877 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-11-02 03:55:48,878 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-11-02 03:55:48,878 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-11-02 03:55:48,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-02 03:55:48,880 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22 2023-11-02 03:55:48,880 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f 2023-11-02 03:55:48,880 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170 2023-11-02 03:55:48,880 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22 empty. 2023-11-02 03:55:48,880 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170 empty. 2023-11-02 03:55:48,880 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f empty. 2023-11-02 03:55:48,881 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22 2023-11-02 03:55:48,881 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170 2023-11-02 03:55:48,881 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f 2023-11-02 03:55:48,881 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-11-02 03:55:48,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-02 03:55:49,180 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-02 03:55:49,287 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-11-02 03:55:49,288 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6aad23f6dcc1d84b230e187242bb5c22, NAME => 'testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10R\xBC\xA9\x93m4\x9FG"\x5CJXyR:\x87"\x10\xCB\xA0\xFE\x99IM\x0E\x9C\x13\xD73{\xAA\x9C\x1Ev*\x10\xF1;\x92#Mx\xFD\x0CnK\x85\x0C#\x80\xC6\xDE2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp 2023-11-02 03:55:49,289 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7675): creating {ENCODED => 93842bec7617596d83d1e604c6d46170, NAME => 'testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10R\xBC\xA9\x93m4\x9FG"\x5CJXyR:\x87"\x10\xCB\xA0\xFE\x99IM\x0E\x9C\x13\xD73{\xAA\x9C\x1Ev*\x10\xF1;\x92#Mx\xFD\x0CnK\x85\x0C#\x80\xC6\xDE2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp 2023-11-02 03:55:49,289 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7675): creating {ENCODED => 8d228d83c8f828dbf92eb31efd23c94f, NAME => 'testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10R\xBC\xA9\x93m4\x9FG"\x5CJXyR:\x87"\x10\xCB\xA0\xFE\x99IM\x0E\x9C\x13\xD73{\xAA\x9C\x1Ev*\x10\xF1;\x92#Mx\xFD\x0CnK\x85\x0C#\x80\xC6\xDE2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/.tmp 2023-11-02 03:55:49,296 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:55:49,296 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1604): Closing 6aad23f6dcc1d84b230e187242bb5c22, disabling compactions & flushes 2023-11-02 03:55:49,296 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:55:49,296 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:55:49,296 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1604): Closing 93842bec7617596d83d1e604c6d46170, disabling compactions & flushes 2023-11-02 03:55:49,296 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:55:49,296 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:55:49,296 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. after waiting 0 ms 2023-11-02 03:55:49,296 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:55:49,296 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:55:49,296 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. after waiting 0 ms 2023-11-02 03:55:49,296 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:55:49,296 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:55:49,296 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1558): Region close journal for 6aad23f6dcc1d84b230e187242bb5c22: 2023-11-02 03:55:49,296 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:55:49,296 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1558): Region close journal for 93842bec7617596d83d1e604c6d46170: 2023-11-02 03:55:49,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-02 03:55:49,695 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:55:49,695 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1604): Closing 8d228d83c8f828dbf92eb31efd23c94f, disabling compactions & flushes 2023-11-02 03:55:49,695 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:55:49,695 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:55:49,695 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. after waiting 0 ms 2023-11-02 03:55:49,695 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:55:49,695 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:55:49,695 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1558): Region close journal for 8d228d83c8f828dbf92eb31efd23c94f: 2023-11-02 03:55:49,697 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-11-02 03:55:49,698 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698897349698"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897349698"}]},"ts":"1698897349698"} 2023-11-02 03:55:49,698 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698897349698"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897349698"}]},"ts":"1698897349698"} 2023-11-02 03:55:49,698 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1698897349698"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1698897349698"}]},"ts":"1698897349698"} 2023-11-02 03:55:49,700 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-11-02 03:55:49,701 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-11-02 03:55:49,701 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897349701"}]},"ts":"1698897349701"} 2023-11-02 03:55:49,702 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-11-02 03:55:49,705 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=6aad23f6dcc1d84b230e187242bb5c22, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=8d228d83c8f828dbf92eb31efd23c94f, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=93842bec7617596d83d1e604c6d46170, ASSIGN}] 2023-11-02 03:55:49,707 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=6aad23f6dcc1d84b230e187242bb5c22, ASSIGN 2023-11-02 03:55:49,707 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=8d228d83c8f828dbf92eb31efd23c94f, ASSIGN 2023-11-02 03:55:49,707 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=93842bec7617596d83d1e604c6d46170, ASSIGN 2023-11-02 03:55:49,707 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=6aad23f6dcc1d84b230e187242bb5c22, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=false 2023-11-02 03:55:49,707 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=8d228d83c8f828dbf92eb31efd23c94f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=false 2023-11-02 03:55:49,707 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=93842bec7617596d83d1e604c6d46170, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36627,1698897201574; forceNewPlan=false, retain=false 2023-11-02 03:55:49,859 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=8d228d83c8f828dbf92eb31efd23c94f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:49,859 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=6aad23f6dcc1d84b230e187242bb5c22, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:49,859 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=93842bec7617596d83d1e604c6d46170, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:49,859 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1698897349859"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897349859"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897349859"}]},"ts":"1698897349859"} 2023-11-02 03:55:49,859 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698897349859"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897349859"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897349859"}]},"ts":"1698897349859"} 2023-11-02 03:55:49,859 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698897349859"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1698897349859"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1698897349859"}]},"ts":"1698897349859"} 2023-11-02 03:55:49,861 INFO [PEWorker-1] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE; OpenRegionProcedure 93842bec7617596d83d1e604c6d46170, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:55:49,861 INFO [PEWorker-2] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=121, ppid=118, state=RUNNABLE; OpenRegionProcedure 8d228d83c8f828dbf92eb31efd23c94f, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:55:49,862 INFO [PEWorker-5] procedure2.ProcedureExecutor(1680): Initialized subprocedures=[{pid=122, ppid=117, state=RUNNABLE; OpenRegionProcedure 6aad23f6dcc1d84b230e187242bb5c22, server=jenkins-hbase4.apache.org,36627,1698897201574}] 2023-11-02 03:55:49,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-02 03:55:50,012 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:50,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:55:50,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6aad23f6dcc1d84b230e187242bb5c22, NAME => 'testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22.', STARTKEY => '', ENDKEY => '1'} 2023-11-02 03:55:50,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 6aad23f6dcc1d84b230e187242bb5c22 2023-11-02 03:55:50,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:55:50,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6aad23f6dcc1d84b230e187242bb5c22 2023-11-02 03:55:50,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6aad23f6dcc1d84b230e187242bb5c22 2023-11-02 03:55:50,037 INFO [StoreOpener-6aad23f6dcc1d84b230e187242bb5c22-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6aad23f6dcc1d84b230e187242bb5c22 2023-11-02 03:55:50,038 DEBUG [StoreOpener-6aad23f6dcc1d84b230e187242bb5c22-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1 2023-11-02 03:55:50,038 DEBUG [StoreOpener-6aad23f6dcc1d84b230e187242bb5c22-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1 2023-11-02 03:55:50,039 INFO [StoreOpener-6aad23f6dcc1d84b230e187242bb5c22-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6aad23f6dcc1d84b230e187242bb5c22 columnFamilyName family1 2023-11-02 03:55:50,040 INFO [StoreOpener-6aad23f6dcc1d84b230e187242bb5c22-1] regionserver.HStore(310): Store=6aad23f6dcc1d84b230e187242bb5c22/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:55:50,040 INFO [StoreOpener-6aad23f6dcc1d84b230e187242bb5c22-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 6aad23f6dcc1d84b230e187242bb5c22 2023-11-02 03:55:50,041 DEBUG [StoreOpener-6aad23f6dcc1d84b230e187242bb5c22-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2 2023-11-02 03:55:50,041 DEBUG [StoreOpener-6aad23f6dcc1d84b230e187242bb5c22-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2 2023-11-02 03:55:50,042 INFO [StoreOpener-6aad23f6dcc1d84b230e187242bb5c22-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6aad23f6dcc1d84b230e187242bb5c22 columnFamilyName family2 2023-11-02 03:55:50,043 INFO [StoreOpener-6aad23f6dcc1d84b230e187242bb5c22-1] regionserver.HStore(310): Store=6aad23f6dcc1d84b230e187242bb5c22/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:55:50,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22 2023-11-02 03:55:50,044 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22 2023-11-02 03:55:50,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-02 03:55:50,046 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6aad23f6dcc1d84b230e187242bb5c22 2023-11-02 03:55:50,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:55:50,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6aad23f6dcc1d84b230e187242bb5c22; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72953254, jitterRate=0.0870881974697113}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-02 03:55:50,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6aad23f6dcc1d84b230e187242bb5c22: 2023-11-02 03:55:50,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22., pid=122, masterSystemTime=1698897350012 2023-11-02 03:55:50,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:55:50,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:55:50,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:55:50,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8d228d83c8f828dbf92eb31efd23c94f, NAME => 'testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f.', STARTKEY => '1', ENDKEY => '2'} 2023-11-02 03:55:50,050 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=6aad23f6dcc1d84b230e187242bb5c22, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:50,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 8d228d83c8f828dbf92eb31efd23c94f 2023-11-02 03:55:50,050 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698897350050"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897350050"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897350050"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897350050"}]},"ts":"1698897350050"} 2023-11-02 03:55:50,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:55:50,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8d228d83c8f828dbf92eb31efd23c94f 2023-11-02 03:55:50,050 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8d228d83c8f828dbf92eb31efd23c94f 2023-11-02 03:55:50,053 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=122, resume processing ppid=117 2023-11-02 03:55:50,053 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=122, ppid=117, state=SUCCESS; OpenRegionProcedure 6aad23f6dcc1d84b230e187242bb5c22, server=jenkins-hbase4.apache.org,36627,1698897201574 in 189 msec 2023-11-02 03:55:50,053 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=6aad23f6dcc1d84b230e187242bb5c22, ASSIGN in 348 msec 2023-11-02 03:55:50,071 INFO [StoreOpener-8d228d83c8f828dbf92eb31efd23c94f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8d228d83c8f828dbf92eb31efd23c94f 2023-11-02 03:55:50,072 DEBUG [StoreOpener-8d228d83c8f828dbf92eb31efd23c94f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1 2023-11-02 03:55:50,072 DEBUG [StoreOpener-8d228d83c8f828dbf92eb31efd23c94f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1 2023-11-02 03:55:50,073 INFO [StoreOpener-8d228d83c8f828dbf92eb31efd23c94f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8d228d83c8f828dbf92eb31efd23c94f columnFamilyName family1 2023-11-02 03:55:50,073 INFO [StoreOpener-8d228d83c8f828dbf92eb31efd23c94f-1] regionserver.HStore(310): Store=8d228d83c8f828dbf92eb31efd23c94f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:55:50,074 INFO [StoreOpener-8d228d83c8f828dbf92eb31efd23c94f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 8d228d83c8f828dbf92eb31efd23c94f 2023-11-02 03:55:50,074 DEBUG [StoreOpener-8d228d83c8f828dbf92eb31efd23c94f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2 2023-11-02 03:55:50,075 DEBUG [StoreOpener-8d228d83c8f828dbf92eb31efd23c94f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2 2023-11-02 03:55:50,075 INFO [StoreOpener-8d228d83c8f828dbf92eb31efd23c94f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8d228d83c8f828dbf92eb31efd23c94f columnFamilyName family2 2023-11-02 03:55:50,075 INFO [StoreOpener-8d228d83c8f828dbf92eb31efd23c94f-1] regionserver.HStore(310): Store=8d228d83c8f828dbf92eb31efd23c94f/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:55:50,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f 2023-11-02 03:55:50,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f 2023-11-02 03:55:50,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-02 03:55:50,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8d228d83c8f828dbf92eb31efd23c94f 2023-11-02 03:55:50,080 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:55:50,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8d228d83c8f828dbf92eb31efd23c94f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69217995, jitterRate=0.0314285010099411}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-02 03:55:50,080 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8d228d83c8f828dbf92eb31efd23c94f: 2023-11-02 03:55:50,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f., pid=121, masterSystemTime=1698897350012 2023-11-02 03:55:50,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:55:50,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:55:50,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:55:50,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 93842bec7617596d83d1e604c6d46170, NAME => 'testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170.', STARTKEY => '2', ENDKEY => ''} 2023-11-02 03:55:50,082 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=8d228d83c8f828dbf92eb31efd23c94f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:50,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 93842bec7617596d83d1e604c6d46170 2023-11-02 03:55:50,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-11-02 03:55:50,082 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1698897350082"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897350082"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897350082"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897350082"}]},"ts":"1698897350082"} 2023-11-02 03:55:50,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 93842bec7617596d83d1e604c6d46170 2023-11-02 03:55:50,083 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 93842bec7617596d83d1e604c6d46170 2023-11-02 03:55:50,085 INFO [PEWorker-4] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=121, resume processing ppid=118 2023-11-02 03:55:50,085 INFO [PEWorker-4] procedure2.ProcedureExecutor(1409): Finished pid=121, ppid=118, state=SUCCESS; OpenRegionProcedure 8d228d83c8f828dbf92eb31efd23c94f, server=jenkins-hbase4.apache.org,36627,1698897201574 in 223 msec 2023-11-02 03:55:50,085 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=8d228d83c8f828dbf92eb31efd23c94f, ASSIGN in 380 msec 2023-11-02 03:55:50,103 INFO [StoreOpener-93842bec7617596d83d1e604c6d46170-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 93842bec7617596d83d1e604c6d46170 2023-11-02 03:55:50,105 DEBUG [StoreOpener-93842bec7617596d83d1e604c6d46170-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1 2023-11-02 03:55:50,105 DEBUG [StoreOpener-93842bec7617596d83d1e604c6d46170-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1 2023-11-02 03:55:50,105 INFO [StoreOpener-93842bec7617596d83d1e604c6d46170-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 93842bec7617596d83d1e604c6d46170 columnFamilyName family1 2023-11-02 03:55:50,106 INFO [StoreOpener-93842bec7617596d83d1e604c6d46170-1] regionserver.HStore(310): Store=93842bec7617596d83d1e604c6d46170/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:55:50,106 INFO [StoreOpener-93842bec7617596d83d1e604c6d46170-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 93842bec7617596d83d1e604c6d46170 2023-11-02 03:55:50,107 DEBUG [StoreOpener-93842bec7617596d83d1e604c6d46170-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2 2023-11-02 03:55:50,107 DEBUG [StoreOpener-93842bec7617596d83d1e604c6d46170-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2 2023-11-02 03:55:50,107 INFO [StoreOpener-93842bec7617596d83d1e604c6d46170-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 93842bec7617596d83d1e604c6d46170 columnFamilyName family2 2023-11-02 03:55:50,108 INFO [StoreOpener-93842bec7617596d83d1e604c6d46170-1] regionserver.HStore(310): Store=93842bec7617596d83d1e604c6d46170/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-11-02 03:55:50,108 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170 2023-11-02 03:55:50,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170 2023-11-02 03:55:50,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-11-02 03:55:50,111 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 93842bec7617596d83d1e604c6d46170 2023-11-02 03:55:50,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-11-02 03:55:50,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 93842bec7617596d83d1e604c6d46170; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60239995, jitterRate=-0.10235412418842316}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-11-02 03:55:50,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 93842bec7617596d83d1e604c6d46170: 2023-11-02 03:55:50,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170., pid=120, masterSystemTime=1698897350012 2023-11-02 03:55:50,115 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:55:50,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:55:50,115 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=93842bec7617596d83d1e604c6d46170, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:50,115 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1698897350115"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1698897350115"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1698897350115"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1698897350115"}]},"ts":"1698897350115"} 2023-11-02 03:55:50,117 INFO [PEWorker-1] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=120, resume processing ppid=119 2023-11-02 03:55:50,117 INFO [PEWorker-1] procedure2.ProcedureExecutor(1409): Finished pid=120, ppid=119, state=SUCCESS; OpenRegionProcedure 93842bec7617596d83d1e604c6d46170, server=jenkins-hbase4.apache.org,36627,1698897201574 in 255 msec 2023-11-02 03:55:50,118 INFO [PEWorker-2] procedure2.ProcedureExecutor(1823): Finished subprocedure pid=119, resume processing ppid=116 2023-11-02 03:55:50,118 INFO [PEWorker-2] procedure2.ProcedureExecutor(1409): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=93842bec7617596d83d1e604c6d46170, ASSIGN in 412 msec 2023-11-02 03:55:50,119 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-11-02 03:55:50,119 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1698897350119"}]},"ts":"1698897350119"} 2023-11-02 03:55:50,120 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-11-02 03:55:50,122 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-11-02 03:55:50,123 INFO [PEWorker-5] procedure2.ProcedureExecutor(1409): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 1.2470 sec 2023-11-02 03:55:50,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1230): Checking to see if procedure is done pid=116 2023-11-02 03:55:50,984 INFO [Listener at localhost/43647] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-11-02 03:55:50,989 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:50,989 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:50,990 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:50,992 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:50,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-11-02 03:55:50,994 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-11-02 03:55:50,994 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:50,994 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-11-02 03:55:50,994 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-11-02 03:55:50,995 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:50,995 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:50,997 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:50,997 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:50,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:50,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:50,997 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:50,997 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:50,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:50,998 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:50,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:50,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:50,998 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-11-02 03:55:50,998 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-11-02 03:55:50,998 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-11-02 03:55:50,998 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:50,998 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-11-02 03:55:50,998 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:55:50,998 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-02 03:55:50,998 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:55:50,998 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:55:50,998 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. started... 2023-11-02 03:55:50,999 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. started... 2023-11-02 03:55:50,999 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. started... 2023-11-02 03:55:50,998 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:51,000 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 93842bec7617596d83d1e604c6d46170 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-02 03:55:51,000 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6aad23f6dcc1d84b230e187242bb5c22 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-02 03:55:51,000 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 8d228d83c8f828dbf92eb31efd23c94f 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-11-02 03:55:51,054 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311023f1b7c4592e44fa4ba2ac142dec48851 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202311023f1b7c4592e44fa4ba2ac142dec48851 2023-11-02 03:55:51,455 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231102e683f3fcd9b0479eb0875de05e4ce622 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231102e683f3fcd9b0479eb0875de05e4ce622 2023-11-02 03:55:51,466 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231102ffbb32101862408389f4da0beba804ff to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231102ffbb32101862408389f4da0beba804ff 2023-11-02 03:55:51,468 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family1/1417e2e07b834944968ecba7eb467de6 2023-11-02 03:55:51,471 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family1/f36aff93a7434c50baee476996d808b3 2023-11-02 03:55:51,490 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family1/f88ce28b5fc748ae9f2eb61a46fbef4a 2023-11-02 03:55:51,515 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231102bec00d2e2f10499a8974e42178868d60 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20231102bec00d2e2f10499a8974e42178868d60 2023-11-02 03:55:51,527 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231102efd047c932d64708a59308f285d8571d to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20231102efd047c932d64708a59308f285d8571d 2023-11-02 03:55:51,530 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family2/b68ae350fea5466aa0f3674d88aef3c7 2023-11-02 03:55:51,533 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family1/f88ce28b5fc748ae9f2eb61a46fbef4a as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/f88ce28b5fc748ae9f2eb61a46fbef4a 2023-11-02 03:55:51,556 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/f88ce28b5fc748ae9f2eb61a46fbef4a, entries=4, sequenceid=5, filesize=5.5 K 2023-11-02 03:55:51,557 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family2/b68ae350fea5466aa0f3674d88aef3c7 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/b68ae350fea5466aa0f3674d88aef3c7 2023-11-02 03:55:51,560 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/b68ae350fea5466aa0f3674d88aef3c7, entries=2, sequenceid=5, filesize=5.1 K 2023-11-02 03:55:51,561 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 6aad23f6dcc1d84b230e187242bb5c22 in 561ms, sequenceid=5, compaction requested=false 2023-11-02 03:55:51,561 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-11-02 03:55:51,561 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6aad23f6dcc1d84b230e187242bb5c22: 2023-11-02 03:55:51,561 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:55:51,561 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-02 03:55:51,908 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311024193e350115745c09933efc260b73e22 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202311024193e350115745c09933efc260b73e22 2023-11-02 03:55:51,920 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family2/f354c3fef34c4b2f97ba59c918c09d8f 2023-11-02 03:55:51,924 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family1/1417e2e07b834944968ecba7eb467de6 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/1417e2e07b834944968ecba7eb467de6 2023-11-02 03:55:51,947 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/1417e2e07b834944968ecba7eb467de6, entries=4, sequenceid=5, filesize=5.5 K 2023-11-02 03:55:51,948 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family2/f354c3fef34c4b2f97ba59c918c09d8f as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/f354c3fef34c4b2f97ba59c918c09d8f 2023-11-02 03:55:51,951 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/f354c3fef34c4b2f97ba59c918c09d8f, entries=2, sequenceid=5, filesize=5.1 K 2023-11-02 03:55:51,952 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 8d228d83c8f828dbf92eb31efd23c94f in 952ms, sequenceid=5, compaction requested=false 2023-11-02 03:55:51,952 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8d228d83c8f828dbf92eb31efd23c94f: 2023-11-02 03:55:51,952 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:55:51,952 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-02 03:55:52,314 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family2/17b203ab9e064e9ab67c4f1a4ea14e5d 2023-11-02 03:55:52,318 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family1/f36aff93a7434c50baee476996d808b3 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/f36aff93a7434c50baee476996d808b3 2023-11-02 03:55:52,351 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/f36aff93a7434c50baee476996d808b3, entries=4, sequenceid=5, filesize=5.5 K 2023-11-02 03:55:52,352 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family2/17b203ab9e064e9ab67c4f1a4ea14e5d as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/17b203ab9e064e9ab67c4f1a4ea14e5d 2023-11-02 03:55:52,355 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/17b203ab9e064e9ab67c4f1a4ea14e5d, entries=2, sequenceid=5, filesize=5.1 K 2023-11-02 03:55:52,356 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 93842bec7617596d83d1e604c6d46170 in 1357ms, sequenceid=5, compaction requested=false 2023-11-02 03:55:52,356 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 93842bec7617596d83d1e604c6d46170: 2023-11-02 03:55:52,356 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:55:52,356 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-02 03:55:52,357 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-02 03:55:52,357 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:52,357 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-11-02 03:55:52,357 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-11-02 03:55:52,358 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:52,358 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:52,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:52,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:52,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:52,359 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:52,359 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:52,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:52,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:52,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:52,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:52,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:52,360 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-11-02 03:55:52,360 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38f916e7[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:52,360 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-11-02 03:55:52,360 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:52,361 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:52,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:52,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:52,361 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-11-02 03:55:52,361 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-11-02 03:55:52,361 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:52,361 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:52,362 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:52,363 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-11-02 03:55:52,363 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:52,363 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:52,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:52,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:52,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:52,363 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-11-02 03:55:52,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:52,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:52,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:52,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:52,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:52,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:52,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:52,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:52,365 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-11-02 03:55:52,365 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-11-02 03:55:52,365 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:52,365 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:52,365 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-11-02 03:55:52,365 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:52,372 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:52,372 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:52,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:52,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:52,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:52,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:52,372 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:52,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:52,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:52,372 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:52,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:52,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:52,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:52,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:52,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:52,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:52,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:52,373 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:52,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:52,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:52,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:52,375 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:52,375 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:52,375 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:52,375 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:52,376 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:52,376 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:52,376 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:52,376 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:52,376 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-11-02 03:55:52,375 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:52,376 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-11-02 03:55:52,376 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:52,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:52,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:52,376 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:52,376 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:52,376 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:52,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:52,376 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:52,476 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:52,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:52,478 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:52,479 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:52,479 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:52,481 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:52,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-11-02 03:55:52,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-11-02 03:55:52,483 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-11-02 03:55:52,483 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:52,483 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-11-02 03:55:52,483 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-11-02 03:55:52,484 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:52,484 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:52,485 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:52,485 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:52,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:52,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:52,485 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:52,485 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:52,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:52,486 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:52,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:52,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:52,486 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-11-02 03:55:52,486 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-11-02 03:55:52,486 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-11-02 03:55:52,486 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-11-02 03:55:52,486 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:52,486 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-11-02 03:55:52,487 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:55:52,487 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. started... 2023-11-02 03:55:52,487 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-02 03:55:52,487 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:52,487 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:55:52,487 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 93842bec7617596d83d1e604c6d46170 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-02 03:55:52,487 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:55:52,487 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. started... 2023-11-02 03:55:52,487 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. started... 2023-11-02 03:55:52,487 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 8d228d83c8f828dbf92eb31efd23c94f 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-02 03:55:52,487 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6aad23f6dcc1d84b230e187242bb5c22 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-11-02 03:55:52,532 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231102c2ebf8fd237d42b88c572dbc24e08482 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231102c2ebf8fd237d42b88c572dbc24e08482 2023-11-02 03:55:52,944 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311023065b46fc2004915a4429b04ea104c56 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202311023065b46fc2004915a4429b04ea104c56 2023-11-02 03:55:52,945 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311020ee657e2472e4d6d87ba06147e0cb623 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202311020ee657e2472e4d6d87ba06147e0cb623 2023-11-02 03:55:52,946 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family1/7a53982970bc4f5e8e47ae3e0ba303c1 2023-11-02 03:55:52,960 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family1/8a005b18883c485aa330a889ad947217 2023-11-02 03:55:53,006 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231102f663263dd2c447499650e29df5214f51 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20231102f663263dd2c447499650e29df5214f51 2023-11-02 03:55:53,014 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family2/a3e22c3eb50a47fa9aee8e4c9f17bf0f 2023-11-02 03:55:53,018 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family1/8a005b18883c485aa330a889ad947217 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/8a005b18883c485aa330a889ad947217 2023-11-02 03:55:53,053 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/8a005b18883c485aa330a889ad947217, entries=4, sequenceid=9, filesize=5.5 K 2023-11-02 03:55:53,054 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family2/a3e22c3eb50a47fa9aee8e4c9f17bf0f as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/a3e22c3eb50a47fa9aee8e4c9f17bf0f 2023-11-02 03:55:53,058 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/a3e22c3eb50a47fa9aee8e4c9f17bf0f, entries=2, sequenceid=9, filesize=5.1 K 2023-11-02 03:55:53,059 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 93842bec7617596d83d1e604c6d46170 in 572ms, sequenceid=9, compaction requested=false 2023-11-02 03:55:53,059 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 93842bec7617596d83d1e604c6d46170: 2023-11-02 03:55:53,059 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:55:53,059 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-02 03:55:53,361 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family1/60bd69509ca34cdf8864e80ef8d4079e 2023-11-02 03:55:53,378 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311028e0eb2734b824c3097c41517a2b968b0 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202311028e0eb2734b824c3097c41517a2b968b0 2023-11-02 03:55:53,382 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family2/b440a8487a27436ba7e36c905e49d51a 2023-11-02 03:55:53,386 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family1/7a53982970bc4f5e8e47ae3e0ba303c1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/7a53982970bc4f5e8e47ae3e0ba303c1 2023-11-02 03:55:53,420 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/7a53982970bc4f5e8e47ae3e0ba303c1, entries=4, sequenceid=9, filesize=5.5 K 2023-11-02 03:55:53,421 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family2/b440a8487a27436ba7e36c905e49d51a as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/b440a8487a27436ba7e36c905e49d51a 2023-11-02 03:55:53,424 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/b440a8487a27436ba7e36c905e49d51a, entries=2, sequenceid=9, filesize=5.1 K 2023-11-02 03:55:53,425 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 8d228d83c8f828dbf92eb31efd23c94f in 938ms, sequenceid=9, compaction requested=false 2023-11-02 03:55:53,425 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8d228d83c8f828dbf92eb31efd23c94f: 2023-11-02 03:55:53,425 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:55:53,425 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-02 03:55:53,794 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231102362c3f40fa06439e885313504f0156a4 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20231102362c3f40fa06439e885313504f0156a4 2023-11-02 03:55:54,198 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family2/8c99bea1d05447e1b35dad5ebe16e3e6 2023-11-02 03:55:54,202 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family1/60bd69509ca34cdf8864e80ef8d4079e as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/60bd69509ca34cdf8864e80ef8d4079e 2023-11-02 03:55:54,225 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/60bd69509ca34cdf8864e80ef8d4079e, entries=4, sequenceid=9, filesize=5.5 K 2023-11-02 03:55:54,226 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family2/8c99bea1d05447e1b35dad5ebe16e3e6 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/8c99bea1d05447e1b35dad5ebe16e3e6 2023-11-02 03:55:54,229 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/8c99bea1d05447e1b35dad5ebe16e3e6, entries=2, sequenceid=9, filesize=5.1 K 2023-11-02 03:55:54,230 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 6aad23f6dcc1d84b230e187242bb5c22 in 1743ms, sequenceid=9, compaction requested=false 2023-11-02 03:55:54,230 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6aad23f6dcc1d84b230e187242bb5c22: 2023-11-02 03:55:54,230 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:55:54,230 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-02 03:55:54,230 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-02 03:55:54,230 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:54,230 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-11-02 03:55:54,230 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-11-02 03:55:54,232 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:54,232 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:54,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:54,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:54,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:54,232 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:54,232 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:54,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:54,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:54,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:54,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:54,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:54,233 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-11-02 03:55:54,234 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3ddf5849[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:54,234 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-11-02 03:55:54,234 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:54,235 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:54,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:54,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:54,235 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-11-02 03:55:54,235 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:54,235 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:54,235 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-11-02 03:55:54,235 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:54,236 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-11-02 03:55:54,236 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:54,236 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:54,236 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-11-02 03:55:54,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:54,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:54,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:54,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:54,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:54,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:54,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:54,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:54,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:54,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:54,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:54,238 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-11-02 03:55:54,238 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-11-02 03:55:54,238 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:54,238 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:54,238 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-11-02 03:55:54,238 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:54,241 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:54,241 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:54,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:54,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:54,241 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:54,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:54,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:54,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:54,241 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:54,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:54,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:54,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:54,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:54,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:54,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:54,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:54,242 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:54,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:54,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:54,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:54,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:54,244 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:54,245 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:54,245 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:54,245 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:54,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:54,245 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-11-02 03:55:54,245 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-11-02 03:55:54,245 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:54,245 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:54,245 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:54,245 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:54,245 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:54,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:54,245 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:54,245 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:54,245 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:54,245 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:54,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:54,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:54,345 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:54,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:54,347 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:54,348 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:54,348 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:54,350 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:54,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-11-02 03:55:54,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-11-02 03:55:54,352 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-11-02 03:55:54,352 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:54,352 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-11-02 03:55:54,352 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-11-02 03:55:54,352 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:54,353 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:54,354 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:54,354 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:54,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:54,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:54,354 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:54,354 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:54,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:54,355 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:54,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:54,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:54,355 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-11-02 03:55:54,355 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-11-02 03:55:54,355 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-11-02 03:55:54,356 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-11-02 03:55:54,356 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:54,356 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-11-02 03:55:54,356 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:55:54,356 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. started... 2023-11-02 03:55:54,356 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 93842bec7617596d83d1e604c6d46170 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-02 03:55:54,357 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-02 03:55:54,357 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:55:54,357 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:55:54,357 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. started... 2023-11-02 03:55:54,357 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:54,357 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. started... 2023-11-02 03:55:54,357 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6aad23f6dcc1d84b230e187242bb5c22 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-02 03:55:54,357 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 8d228d83c8f828dbf92eb31efd23c94f 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-11-02 03:55:54,410 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231102a8bf95ca22744b95884535fa59314a93 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231102a8bf95ca22744b95884535fa59314a93 2023-11-02 03:55:54,810 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231102abe95fad387a463eb60e8b5c749efc33 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231102abe95fad387a463eb60e8b5c749efc33 2023-11-02 03:55:54,810 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231102dab4c893958d4cf8a0e61351c394891a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231102dab4c893958d4cf8a0e61351c394891a 2023-11-02 03:55:54,825 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family1/04d99b769dd346bf92c8c373d0bb3a38 2023-11-02 03:55:55,224 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family1/6b656ffa85334e9a80a02bc11ab2f1f3 2023-11-02 03:55:55,225 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family1/4c0caab335d041db99ef8613174b2083 2023-11-02 03:55:55,257 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023110295ba52dc26644ffa893ae348509a8751 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023110295ba52dc26644ffa893ae348509a8751 2023-11-02 03:55:55,259 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023110292416212585841dfaede65bd08845975 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023110292416212585841dfaede65bd08845975 2023-11-02 03:55:55,264 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family2/8bc5fc2d9772487eacbee60e48b3e715 2023-11-02 03:55:55,266 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231102e6613b2c469f413e86d9fa69cf4ba935 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20231102e6613b2c469f413e86d9fa69cf4ba935 2023-11-02 03:55:55,271 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family1/4c0caab335d041db99ef8613174b2083 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/4c0caab335d041db99ef8613174b2083 2023-11-02 03:55:55,278 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family2/913829ef50974742a63bd3fb74d51327 2023-11-02 03:55:55,283 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family1/6b656ffa85334e9a80a02bc11ab2f1f3 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/6b656ffa85334e9a80a02bc11ab2f1f3 2023-11-02 03:55:55,299 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/4c0caab335d041db99ef8613174b2083, entries=4, sequenceid=13, filesize=5.5 K 2023-11-02 03:55:55,300 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family2/8bc5fc2d9772487eacbee60e48b3e715 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/8bc5fc2d9772487eacbee60e48b3e715 2023-11-02 03:55:55,304 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/8bc5fc2d9772487eacbee60e48b3e715, entries=2, sequenceid=13, filesize=5.1 K 2023-11-02 03:55:55,305 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 8d228d83c8f828dbf92eb31efd23c94f in 948ms, sequenceid=13, compaction requested=true 2023-11-02 03:55:55,305 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8d228d83c8f828dbf92eb31efd23c94f: 2023-11-02 03:55:55,305 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:55:55,305 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-02 03:55:55,307 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/6b656ffa85334e9a80a02bc11ab2f1f3, entries=4, sequenceid=13, filesize=5.5 K 2023-11-02 03:55:55,307 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family2/913829ef50974742a63bd3fb74d51327 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/913829ef50974742a63bd3fb74d51327 2023-11-02 03:55:55,311 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/913829ef50974742a63bd3fb74d51327, entries=2, sequenceid=13, filesize=5.1 K 2023-11-02 03:55:55,312 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 93842bec7617596d83d1e604c6d46170 in 956ms, sequenceid=13, compaction requested=true 2023-11-02 03:55:55,312 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 93842bec7617596d83d1e604c6d46170: 2023-11-02 03:55:55,312 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:55:55,312 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-02 03:55:55,556 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-11-02 03:55:55,663 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family2/a3a975da1da8484dbcda5812e11ad0f4 2023-11-02 03:55:55,667 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family1/04d99b769dd346bf92c8c373d0bb3a38 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/04d99b769dd346bf92c8c373d0bb3a38 2023-11-02 03:55:55,700 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/04d99b769dd346bf92c8c373d0bb3a38, entries=4, sequenceid=13, filesize=5.5 K 2023-11-02 03:55:55,701 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family2/a3a975da1da8484dbcda5812e11ad0f4 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/a3a975da1da8484dbcda5812e11ad0f4 2023-11-02 03:55:55,705 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/a3a975da1da8484dbcda5812e11ad0f4, entries=2, sequenceid=13, filesize=5.1 K 2023-11-02 03:55:55,706 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 6aad23f6dcc1d84b230e187242bb5c22 in 1349ms, sequenceid=13, compaction requested=true 2023-11-02 03:55:55,706 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6aad23f6dcc1d84b230e187242bb5c22: 2023-11-02 03:55:55,706 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:55:55,706 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-02 03:55:55,706 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-02 03:55:55,706 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:55,706 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-11-02 03:55:55,706 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-11-02 03:55:55,709 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:55,709 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:55,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:55,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:55,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:55,710 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:55,710 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:55,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:55,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:55,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:55,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:55,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:55,711 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-11-02 03:55:55,711 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@617ef28b[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:55,711 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-11-02 03:55:55,711 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:55,712 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:55,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:55,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:55,712 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-11-02 03:55:55,713 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-11-02 03:55:55,713 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:55,713 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:55,713 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:55,714 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-11-02 03:55:55,714 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:55,714 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:55,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:55,714 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-11-02 03:55:55,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:55,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:55,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:55,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:55,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:55,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:55,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:55,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:55,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:55,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:55,717 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-11-02 03:55:55,717 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-11-02 03:55:55,717 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:55,717 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:55,717 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-11-02 03:55:55,717 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:55,719 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:55,719 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:55,719 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:55,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:55,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:55,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:55,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:55,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:55,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:55,719 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:55,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:55,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:55,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:55,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:55,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:55,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:55,721 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:55,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:55,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:55,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:55,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:55,724 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:55,724 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:55,724 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:55,724 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:55,724 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-11-02 03:55:55,724 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:55,724 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:55,724 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-11-02 03:55:55,724 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:55,724 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:55,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:55,724 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:55,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:55,724 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:55,724 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:55,724 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:55,725 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:55,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:55,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:55,825 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:55,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:55,827 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:55,827 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:55,827 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(8805): writing data to region testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. with WAL disabled. Data may be lost in the event of a crash. 2023-11-02 03:55:55,829 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:55,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-11-02 03:55:55,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-11-02 03:55:55,832 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-11-02 03:55:55,832 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:55,832 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-11-02 03:55:55,832 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-11-02 03:55:55,833 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:55,833 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:55,834 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:55,834 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:55,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:55,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:55,834 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:55,834 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:55,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:55,835 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:55,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:55,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:55,835 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-11-02 03:55:55,835 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-11-02 03:55:55,835 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-11-02 03:55:55,835 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-11-02 03:55:55,836 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:55,836 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-11-02 03:55:55,836 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:55:55,836 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. started... 2023-11-02 03:55:55,836 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:55:55,836 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-02 03:55:55,836 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:55:55,836 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:55,836 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 93842bec7617596d83d1e604c6d46170 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-02 03:55:55,836 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. started... 2023-11-02 03:55:55,836 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. started... 2023-11-02 03:55:55,836 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 8d228d83c8f828dbf92eb31efd23c94f 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-02 03:55:55,836 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6aad23f6dcc1d84b230e187242bb5c22 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-11-02 03:55:55,880 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231102a4bb97cc1c4b4b5abe66b65941226c95 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231102a4bb97cc1c4b4b5abe66b65941226c95 2023-11-02 03:55:55,883 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023110267181fd1df1f4eba8f14f77c90574208 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023110267181fd1df1f4eba8f14f77c90574208 2023-11-02 03:55:55,902 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-02 03:55:55,902 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x749c8902 to 127.0.0.1:65185 2023-11-02 03:55:55,902 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:55:55,903 INFO [Finalizer] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-02 03:55:55,903 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5f35c3cc to 127.0.0.1:65185 2023-11-02 03:55:55,903 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:55:56,283 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023110218aaed54c87b499887bc5a77cc2b2811 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023110218aaed54c87b499887bc5a77cc2b2811 2023-11-02 03:55:56,319 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family1/a5ce1086f65f4f96a434555280e6ab1b 2023-11-02 03:55:56,320 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family1/979b1acfab8b439a8b9b5ed269402035 2023-11-02 03:55:56,698 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family1/a67948688f6343f1931d34b9bdbd4a80 2023-11-02 03:55:56,753 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311020ecd06c46dd64c4d93a3293122b481a8 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202311020ecd06c46dd64c4d93a3293122b481a8 2023-11-02 03:55:56,765 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231102e2601d5f0fb7450286f7b369e47178b1 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20231102e2601d5f0fb7450286f7b369e47178b1 2023-11-02 03:55:56,769 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family2/10fe9b6d1e004888807871e9928a1755 2023-11-02 03:55:56,772 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family1/979b1acfab8b439a8b9b5ed269402035 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/979b1acfab8b439a8b9b5ed269402035 2023-11-02 03:55:56,795 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/979b1acfab8b439a8b9b5ed269402035, entries=4, sequenceid=17, filesize=5.5 K 2023-11-02 03:55:56,795 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family2/10fe9b6d1e004888807871e9928a1755 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/10fe9b6d1e004888807871e9928a1755 2023-11-02 03:55:56,798 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/10fe9b6d1e004888807871e9928a1755, entries=2, sequenceid=17, filesize=5.1 K 2023-11-02 03:55:56,799 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 8d228d83c8f828dbf92eb31efd23c94f in 963ms, sequenceid=17, compaction requested=true 2023-11-02 03:55:56,799 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8d228d83c8f828dbf92eb31efd23c94f: 2023-11-02 03:55:56,799 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:55:56,799 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-02 03:55:57,130 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231102f805dd166b414f7e9f0dd54d9f1ff753 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20231102f805dd166b414f7e9f0dd54d9f1ff753 2023-11-02 03:55:57,157 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family2/1edda564f3014ea1a955d027f32564de 2023-11-02 03:55:57,161 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family1/a5ce1086f65f4f96a434555280e6ab1b as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/a5ce1086f65f4f96a434555280e6ab1b 2023-11-02 03:55:57,196 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/a5ce1086f65f4f96a434555280e6ab1b, entries=4, sequenceid=17, filesize=5.5 K 2023-11-02 03:55:57,197 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family2/1edda564f3014ea1a955d027f32564de as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/1edda564f3014ea1a955d027f32564de 2023-11-02 03:55:57,201 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/1edda564f3014ea1a955d027f32564de, entries=2, sequenceid=17, filesize=5.1 K 2023-11-02 03:55:57,202 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 93842bec7617596d83d1e604c6d46170 in 1366ms, sequenceid=17, compaction requested=true 2023-11-02 03:55:57,202 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 93842bec7617596d83d1e604c6d46170: 2023-11-02 03:55:57,202 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:55:57,202 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-02 03:55:57,535 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family2/f08476af1202463fb61f227b1d1c42c1 2023-11-02 03:55:57,539 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family1/a67948688f6343f1931d34b9bdbd4a80 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/a67948688f6343f1931d34b9bdbd4a80 2023-11-02 03:55:57,561 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/a67948688f6343f1931d34b9bdbd4a80, entries=4, sequenceid=17, filesize=5.5 K 2023-11-02 03:55:57,562 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family2/f08476af1202463fb61f227b1d1c42c1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/f08476af1202463fb61f227b1d1c42c1 2023-11-02 03:55:57,566 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/f08476af1202463fb61f227b1d1c42c1, entries=2, sequenceid=17, filesize=5.1 K 2023-11-02 03:55:57,566 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 6aad23f6dcc1d84b230e187242bb5c22 in 1730ms, sequenceid=17, compaction requested=true 2023-11-02 03:55:57,566 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6aad23f6dcc1d84b230e187242bb5c22: 2023-11-02 03:55:57,566 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:55:57,566 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-02 03:55:57,567 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-02 03:55:57,567 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:57,567 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-11-02 03:55:57,567 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-11-02 03:55:57,568 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:57,568 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:57,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:57,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:57,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:57,569 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:57,569 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:57,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:57,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:57,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:57,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:57,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:57,570 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-11-02 03:55:57,570 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@29f8c0ed[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:57,570 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-11-02 03:55:57,571 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:57,576 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:57,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:57,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:57,576 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-11-02 03:55:57,576 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-11-02 03:55:57,576 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:57,576 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:57,576 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:57,577 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-11-02 03:55:57,577 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:57,577 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:57,577 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-11-02 03:55:57,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:57,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:57,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:57,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:57,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:57,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:57,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:57,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:57,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:57,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:57,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:57,579 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-11-02 03:55:57,579 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-11-02 03:55:57,579 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:57,580 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:57,580 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-11-02 03:55:57,580 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:57,581 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:57,581 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:57,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:57,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:57,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:57,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:57,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:57,581 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:57,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:57,581 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:57,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:57,582 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:57,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:57,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:57,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:57,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:57,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:57,583 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:57,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:57,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:57,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:57,585 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:57,585 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:57,585 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:57,585 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:57,585 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:57,585 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:57,585 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-11-02 03:55:57,585 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:57,585 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:57,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:57,585 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:57,585 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:57,585 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:57,586 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:57,586 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:57,586 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:57,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:57,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:57,686 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:57,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:57,695 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-02 03:55:57,701 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-02 03:55:57,708 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-02 03:55:57,708 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-02 03:55:57,809 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(936): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-11-02 03:55:57,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-11-02 03:55:57,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-11-02 03:55:57,817 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-11-02 03:55:57,817 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:57,817 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-11-02 03:55:57,817 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-11-02 03:55:57,818 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:57,818 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:57,821 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:57,821 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:57,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:57,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:57,822 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:57,822 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-11-02 03:55:57,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:57,822 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:57,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-11-02 03:55:57,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:57,823 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-11-02 03:55:57,823 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-11-02 03:55:57,823 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-11-02 03:55:57,823 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-11-02 03:55:57,823 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-11-02 03:55:57,823 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-11-02 03:55:57,823 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:55:57,823 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. started... 2023-11-02 03:55:57,823 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-11-02 03:55:57,823 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-11-02 03:55:57,823 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:55:57,823 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 93842bec7617596d83d1e604c6d46170 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-02 03:55:57,823 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:55:57,823 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. started... 2023-11-02 03:55:57,824 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. started... 2023-11-02 03:55:57,824 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 8d228d83c8f828dbf92eb31efd23c94f 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-02 03:55:57,824 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6aad23f6dcc1d84b230e187242bb5c22 2/2 column families, dataSize=138 B heapSize=944 B 2023-11-02 03:55:57,863 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family1/10d60d37300144f48bad05f4870891cc 2023-11-02 03:55:57,885 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 10d60d37300144f48bad05f4870891cc 2023-11-02 03:55:58,669 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family1/5857c1f91c574512b896c901fd721392 2023-11-02 03:55:58,669 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family1/90ce131dc98c4f47ada5908c9200fca9 2023-11-02 03:55:58,692 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5857c1f91c574512b896c901fd721392 2023-11-02 03:55:58,692 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 90ce131dc98c4f47ada5908c9200fca9 2023-11-02 03:55:58,695 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family2/513734a30eaf4441bb6166a0418e964c 2023-11-02 03:55:58,699 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 513734a30eaf4441bb6166a0418e964c 2023-11-02 03:55:58,700 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family1/10d60d37300144f48bad05f4870891cc as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/10d60d37300144f48bad05f4870891cc 2023-11-02 03:55:58,722 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 10d60d37300144f48bad05f4870891cc 2023-11-02 03:55:58,722 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/10d60d37300144f48bad05f4870891cc, entries=3, sequenceid=23, filesize=5.2 K 2023-11-02 03:55:58,723 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family2/513734a30eaf4441bb6166a0418e964c as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/513734a30eaf4441bb6166a0418e964c 2023-11-02 03:55:58,725 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 513734a30eaf4441bb6166a0418e964c 2023-11-02 03:55:58,726 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/513734a30eaf4441bb6166a0418e964c, entries=1, sequenceid=23, filesize=5.0 K 2023-11-02 03:55:58,726 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 93842bec7617596d83d1e604c6d46170 in 903ms, sequenceid=23, compaction requested=true 2023-11-02 03:55:58,726 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 93842bec7617596d83d1e604c6d46170: 2023-11-02 03:55:58,726 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:55:58,726 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-11-02 03:55:59,103 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family2/f3f2ff453221472ab096046fb913b382 2023-11-02 03:55:59,106 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f3f2ff453221472ab096046fb913b382 2023-11-02 03:55:59,106 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family1/5857c1f91c574512b896c901fd721392 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/5857c1f91c574512b896c901fd721392 2023-11-02 03:55:59,129 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5857c1f91c574512b896c901fd721392 2023-11-02 03:55:59,129 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/5857c1f91c574512b896c901fd721392, entries=3, sequenceid=23, filesize=5.2 K 2023-11-02 03:55:59,130 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family2/f3f2ff453221472ab096046fb913b382 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/f3f2ff453221472ab096046fb913b382 2023-11-02 03:55:59,133 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f3f2ff453221472ab096046fb913b382 2023-11-02 03:55:59,133 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/f3f2ff453221472ab096046fb913b382, entries=1, sequenceid=23, filesize=5.0 K 2023-11-02 03:55:59,134 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 8d228d83c8f828dbf92eb31efd23c94f in 1310ms, sequenceid=23, compaction requested=true 2023-11-02 03:55:59,134 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 8d228d83c8f828dbf92eb31efd23c94f: 2023-11-02 03:55:59,134 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:55:59,134 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-11-02 03:55:59,504 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family2/f7dd9cbe0f5a4b14b5be5222dc9b0720 2023-11-02 03:55:59,507 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f7dd9cbe0f5a4b14b5be5222dc9b0720 2023-11-02 03:55:59,508 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family1/90ce131dc98c4f47ada5908c9200fca9 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/90ce131dc98c4f47ada5908c9200fca9 2023-11-02 03:55:59,530 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 90ce131dc98c4f47ada5908c9200fca9 2023-11-02 03:55:59,530 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/90ce131dc98c4f47ada5908c9200fca9, entries=3, sequenceid=23, filesize=5.2 K 2023-11-02 03:55:59,531 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family2/f7dd9cbe0f5a4b14b5be5222dc9b0720 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/f7dd9cbe0f5a4b14b5be5222dc9b0720 2023-11-02 03:55:59,534 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f7dd9cbe0f5a4b14b5be5222dc9b0720 2023-11-02 03:55:59,534 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/f7dd9cbe0f5a4b14b5be5222dc9b0720, entries=1, sequenceid=23, filesize=5.0 K 2023-11-02 03:55:59,534 INFO [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 6aad23f6dcc1d84b230e187242bb5c22 in 1710ms, sequenceid=23, compaction requested=true 2023-11-02 03:55:59,534 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6aad23f6dcc1d84b230e187242bb5c22: 2023-11-02 03:55:59,534 DEBUG [rs(jenkins-hbase4.apache.org,36627,1698897201574)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:55:59,535 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-11-02 03:55:59,535 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-11-02 03:55:59,535 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:59,535 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-11-02 03:55:59,535 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-11-02 03:55:59,536 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:59,536 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:59,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:59,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:59,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:59,537 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:59,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:59,537 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-11-02 03:55:59,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:59,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:59,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:59,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:59,538 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36627,1698897201574' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-11-02 03:55:59,538 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5946a51b[Count = 0] remaining members to acquire global barrier 2023-11-02 03:55:59,538 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-11-02 03:55:59,538 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:59,539 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:59,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:59,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:59,539 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-11-02 03:55:59,539 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-11-02 03:55:59,539 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,36627,1698897201574' in zk 2023-11-02 03:55:59,539 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:59,539 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-11-02 03:55:59,541 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-11-02 03:55:59,541 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:59,541 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:59,541 DEBUG [member: 'jenkins-hbase4.apache.org,36627,1698897201574' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-11-02 03:55:59,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:59,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:59,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:59,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:59,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:59,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:59,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:59,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:59,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:59,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:59,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,36627,1698897201574': 2023-11-02 03:55:59,543 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36627,1698897201574' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-11-02 03:55:59,543 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-11-02 03:55:59,543 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-11-02 03:55:59,543 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-11-02 03:55:59,543 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-11-02 03:55:59,543 INFO [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-11-02 03:55:59,545 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:59,545 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:59,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:59,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:59,545 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:59,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:59,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-11-02 03:55:59,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-11-02 03:55:59,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:59,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:59,546 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:59,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-11-02 03:55:59,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:59,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:59,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-11-02 03:55:59,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:59,547 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:59,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:59,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-11-02 03:55:59,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-11-02 03:55:59,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:59,549 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:59,549 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-11-02 03:55:59,549 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:59,549 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-11-02 03:55:59,549 DEBUG [(jenkins-hbase4.apache.org,41275,1698897200405)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-11-02 03:55:59,549 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-11-02 03:55:59,549 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:55:59,549 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:59,549 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-11-02 03:55:59,549 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-11-02 03:55:59,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-11-02 03:55:59,549 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-11-02 03:55:59,549 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-11-02 03:55:59,549 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-11-02 03:55:59,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-11-02 03:55:59,550 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-11-02 03:55:59,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-11-02 03:55:59,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-11-02 03:55:59,550 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-11-02 03:55:59,650 DEBUG [Listener at localhost/43647] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-11-02 03:55:59,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41275] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-11-02 03:55:59,651 DEBUG [Listener at localhost/43647] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-02 03:55:59,652 DEBUG [Listener at localhost/43647] regionserver.HStore(1912): 93842bec7617596d83d1e604c6d46170/family1 is initiating major compaction (all files) 2023-11-02 03:55:59,652 INFO [Listener at localhost/43647] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-02 03:55:59,652 INFO [Listener at localhost/43647] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-02 03:55:59,653 INFO [Listener at localhost/43647] regionserver.HRegion(2259): Starting compaction of 93842bec7617596d83d1e604c6d46170/family1 in testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:55:59,653 INFO [Listener at localhost/43647] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/f36aff93a7434c50baee476996d808b3, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/8a005b18883c485aa330a889ad947217, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/6b656ffa85334e9a80a02bc11ab2f1f3, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/a5ce1086f65f4f96a434555280e6ab1b, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/10d60d37300144f48bad05f4870891cc] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp, totalSize=27.1 K 2023-11-02 03:55:59,654 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting f36aff93a7434c50baee476996d808b3, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897350990 2023-11-02 03:55:59,654 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 8a005b18883c485aa330a889ad947217, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897352479 2023-11-02 03:55:59,654 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 6b656ffa85334e9a80a02bc11ab2f1f3, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897354348 2023-11-02 03:55:59,655 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting a5ce1086f65f4f96a434555280e6ab1b, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897355827 2023-11-02 03:55:59,655 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 10d60d37300144f48bad05f4870891cc, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-02 03:55:59,663 INFO [Listener at localhost/43647] throttle.PressureAwareThroughputController(145): 93842bec7617596d83d1e604c6d46170#family1#compaction#197 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:56:00,116 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202311023b9ce4666fb94b63812b53cc253133b5_del 2023-11-02 03:56:00,116 INFO [Listener at localhost/43647] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202311023b9ce4666fb94b63812b53cc253133b5_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202311023b9ce4666fb94b63812b53cc253133b5_del 2023-11-02 03:56:00,968 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cd5a930a10944a19a49cbe95c93d85e6 2023-11-02 03:56:00,969 DEBUG [Listener at localhost/43647] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family1/cd5a930a10944a19a49cbe95c93d85e6 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/cd5a930a10944a19a49cbe95c93d85e6 2023-11-02 03:56:00,991 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cd5a930a10944a19a49cbe95c93d85e6 2023-11-02 03:56:00,991 INFO [Listener at localhost/43647] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 93842bec7617596d83d1e604c6d46170/family1 of 93842bec7617596d83d1e604c6d46170 into cd5a930a10944a19a49cbe95c93d85e6(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-02 03:56:00,992 DEBUG [Listener at localhost/43647] regionserver.HRegion(2289): Compaction status journal for 93842bec7617596d83d1e604c6d46170: 2023-11-02 03:56:00,992 DEBUG [Listener at localhost/43647] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-02 03:56:00,992 DEBUG [Listener at localhost/43647] regionserver.HStore(1912): 93842bec7617596d83d1e604c6d46170/family2 is initiating major compaction (all files) 2023-11-02 03:56:00,992 INFO [Listener at localhost/43647] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-02 03:56:00,992 INFO [Listener at localhost/43647] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-02 03:56:00,992 INFO [Listener at localhost/43647] regionserver.HRegion(2259): Starting compaction of 93842bec7617596d83d1e604c6d46170/family2 in testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:56:00,992 INFO [Listener at localhost/43647] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/17b203ab9e064e9ab67c4f1a4ea14e5d, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/a3e22c3eb50a47fa9aee8e4c9f17bf0f, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/913829ef50974742a63bd3fb74d51327, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/1edda564f3014ea1a955d027f32564de, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/513734a30eaf4441bb6166a0418e964c] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp, totalSize=25.4 K 2023-11-02 03:56:00,993 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 17b203ab9e064e9ab67c4f1a4ea14e5d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897350990 2023-11-02 03:56:00,993 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting a3e22c3eb50a47fa9aee8e4c9f17bf0f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897352479 2023-11-02 03:56:00,993 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 913829ef50974742a63bd3fb74d51327, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897354348 2023-11-02 03:56:00,994 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 1edda564f3014ea1a955d027f32564de, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897355827 2023-11-02 03:56:00,994 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 513734a30eaf4441bb6166a0418e964c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-02 03:56:01,001 INFO [Listener at localhost/43647] throttle.PressureAwareThroughputController(145): 93842bec7617596d83d1e604c6d46170#family2#compaction#198 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:56:01,003 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/f36aff93a7434c50baee476996d808b3, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/8a005b18883c485aa330a889ad947217, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/6b656ffa85334e9a80a02bc11ab2f1f3, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/a5ce1086f65f4f96a434555280e6ab1b, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/10d60d37300144f48bad05f4870891cc] to archive 2023-11-02 03:56:01,004 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:56:01,005 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/f36aff93a7434c50baee476996d808b3 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/f36aff93a7434c50baee476996d808b3 2023-11-02 03:56:01,006 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/8a005b18883c485aa330a889ad947217 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/8a005b18883c485aa330a889ad947217 2023-11-02 03:56:01,007 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/6b656ffa85334e9a80a02bc11ab2f1f3 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/6b656ffa85334e9a80a02bc11ab2f1f3 2023-11-02 03:56:01,008 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/a5ce1086f65f4f96a434555280e6ab1b to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/a5ce1086f65f4f96a434555280e6ab1b 2023-11-02 03:56:01,009 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20231102a6d76f7712394de09cd225b178c19c85_del 2023-11-02 03:56:01,009 INFO [Listener at localhost/43647] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231102a6d76f7712394de09cd225b178c19c85_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20231102a6d76f7712394de09cd225b178c19c85_del 2023-11-02 03:56:01,009 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/10d60d37300144f48bad05f4870891cc to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/10d60d37300144f48bad05f4870891cc 2023-11-02 03:56:01,821 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 99bb9124ce9f44d8a4754f22a5105fae 2023-11-02 03:56:01,821 DEBUG [Listener at localhost/43647] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/.tmp/family2/99bb9124ce9f44d8a4754f22a5105fae as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/99bb9124ce9f44d8a4754f22a5105fae 2023-11-02 03:56:01,824 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 99bb9124ce9f44d8a4754f22a5105fae 2023-11-02 03:56:01,825 INFO [Listener at localhost/43647] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 93842bec7617596d83d1e604c6d46170/family2 of 93842bec7617596d83d1e604c6d46170 into 99bb9124ce9f44d8a4754f22a5105fae(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-11-02 03:56:01,825 DEBUG [Listener at localhost/43647] regionserver.HRegion(2289): Compaction status journal for 93842bec7617596d83d1e604c6d46170: 2023-11-02 03:56:01,825 DEBUG [Listener at localhost/43647] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-02 03:56:01,825 DEBUG [Listener at localhost/43647] regionserver.HStore(1912): 8d228d83c8f828dbf92eb31efd23c94f/family1 is initiating major compaction (all files) 2023-11-02 03:56:01,826 INFO [Listener at localhost/43647] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-02 03:56:01,826 INFO [Listener at localhost/43647] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-02 03:56:01,826 INFO [Listener at localhost/43647] regionserver.HRegion(2259): Starting compaction of 8d228d83c8f828dbf92eb31efd23c94f/family1 in testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:56:01,826 INFO [Listener at localhost/43647] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/1417e2e07b834944968ecba7eb467de6, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/7a53982970bc4f5e8e47ae3e0ba303c1, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/4c0caab335d041db99ef8613174b2083, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/979b1acfab8b439a8b9b5ed269402035, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/5857c1f91c574512b896c901fd721392] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp, totalSize=27.1 K 2023-11-02 03:56:01,826 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 1417e2e07b834944968ecba7eb467de6, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897350989 2023-11-02 03:56:01,827 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 7a53982970bc4f5e8e47ae3e0ba303c1, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897352479 2023-11-02 03:56:01,827 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 4c0caab335d041db99ef8613174b2083, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897354348 2023-11-02 03:56:01,827 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 979b1acfab8b439a8b9b5ed269402035, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897355827 2023-11-02 03:56:01,828 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 5857c1f91c574512b896c901fd721392, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-02 03:56:01,835 INFO [Listener at localhost/43647] throttle.PressureAwareThroughputController(145): 8d228d83c8f828dbf92eb31efd23c94f#family1#compaction#199 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:56:01,904 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/17b203ab9e064e9ab67c4f1a4ea14e5d, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/a3e22c3eb50a47fa9aee8e4c9f17bf0f, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/913829ef50974742a63bd3fb74d51327, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/1edda564f3014ea1a955d027f32564de, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/513734a30eaf4441bb6166a0418e964c] to archive 2023-11-02 03:56:01,905 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:56:01,906 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/17b203ab9e064e9ab67c4f1a4ea14e5d to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/17b203ab9e064e9ab67c4f1a4ea14e5d 2023-11-02 03:56:01,907 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/a3e22c3eb50a47fa9aee8e4c9f17bf0f to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/a3e22c3eb50a47fa9aee8e4c9f17bf0f 2023-11-02 03:56:01,908 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/913829ef50974742a63bd3fb74d51327 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/913829ef50974742a63bd3fb74d51327 2023-11-02 03:56:01,909 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/1edda564f3014ea1a955d027f32564de to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/1edda564f3014ea1a955d027f32564de 2023-11-02 03:56:01,910 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/513734a30eaf4441bb6166a0418e964c to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family2/513734a30eaf4441bb6166a0418e964c 2023-11-02 03:56:02,275 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231102f08d4fdde4cb44af8aca8064c6c8474f_del 2023-11-02 03:56:02,275 INFO [Listener at localhost/43647] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231102f08d4fdde4cb44af8aca8064c6c8474f_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231102f08d4fdde4cb44af8aca8064c6c8474f_del 2023-11-02 03:56:03,128 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 167e8847686e40b28d804aab07376272 2023-11-02 03:56:03,129 DEBUG [Listener at localhost/43647] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family1/167e8847686e40b28d804aab07376272 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/167e8847686e40b28d804aab07376272 2023-11-02 03:56:03,151 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 167e8847686e40b28d804aab07376272 2023-11-02 03:56:03,152 INFO [Listener at localhost/43647] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 8d228d83c8f828dbf92eb31efd23c94f/family1 of 8d228d83c8f828dbf92eb31efd23c94f into 167e8847686e40b28d804aab07376272(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-02 03:56:03,152 DEBUG [Listener at localhost/43647] regionserver.HRegion(2289): Compaction status journal for 8d228d83c8f828dbf92eb31efd23c94f: 2023-11-02 03:56:03,152 DEBUG [Listener at localhost/43647] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-02 03:56:03,152 DEBUG [Listener at localhost/43647] regionserver.HStore(1912): 8d228d83c8f828dbf92eb31efd23c94f/family2 is initiating major compaction (all files) 2023-11-02 03:56:03,152 INFO [Listener at localhost/43647] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-02 03:56:03,152 INFO [Listener at localhost/43647] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-02 03:56:03,154 INFO [Listener at localhost/43647] regionserver.HRegion(2259): Starting compaction of 8d228d83c8f828dbf92eb31efd23c94f/family2 in testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:56:03,154 INFO [Listener at localhost/43647] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/f354c3fef34c4b2f97ba59c918c09d8f, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/b440a8487a27436ba7e36c905e49d51a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/8bc5fc2d9772487eacbee60e48b3e715, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/10fe9b6d1e004888807871e9928a1755, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/f3f2ff453221472ab096046fb913b382] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp, totalSize=25.4 K 2023-11-02 03:56:03,154 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting f354c3fef34c4b2f97ba59c918c09d8f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897350989 2023-11-02 03:56:03,155 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting b440a8487a27436ba7e36c905e49d51a, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897352479 2023-11-02 03:56:03,155 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 8bc5fc2d9772487eacbee60e48b3e715, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897354348 2023-11-02 03:56:03,155 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 10fe9b6d1e004888807871e9928a1755, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897355827 2023-11-02 03:56:03,156 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting f3f2ff453221472ab096046fb913b382, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-02 03:56:03,163 INFO [Listener at localhost/43647] throttle.PressureAwareThroughputController(145): 8d228d83c8f828dbf92eb31efd23c94f#family2#compaction#200 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:56:03,205 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/1417e2e07b834944968ecba7eb467de6, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/7a53982970bc4f5e8e47ae3e0ba303c1, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/4c0caab335d041db99ef8613174b2083, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/979b1acfab8b439a8b9b5ed269402035, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/5857c1f91c574512b896c901fd721392] to archive 2023-11-02 03:56:03,206 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:56:03,207 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/1417e2e07b834944968ecba7eb467de6 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/1417e2e07b834944968ecba7eb467de6 2023-11-02 03:56:03,208 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/7a53982970bc4f5e8e47ae3e0ba303c1 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/7a53982970bc4f5e8e47ae3e0ba303c1 2023-11-02 03:56:03,209 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/4c0caab335d041db99ef8613174b2083 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/4c0caab335d041db99ef8613174b2083 2023-11-02 03:56:03,210 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/979b1acfab8b439a8b9b5ed269402035 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/979b1acfab8b439a8b9b5ed269402035 2023-11-02 03:56:03,211 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/5857c1f91c574512b896c901fd721392 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/5857c1f91c574512b896c901fd721392 2023-11-02 03:56:03,569 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231102174e58c8e8384abfa4ea80b4e481bf8f_del 2023-11-02 03:56:03,570 INFO [Listener at localhost/43647] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20231102174e58c8e8384abfa4ea80b4e481bf8f_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20231102174e58c8e8384abfa4ea80b4e481bf8f_del 2023-11-02 03:56:04,383 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 718f9b0dd3454385a76ef2be7aeedd9f 2023-11-02 03:56:04,384 DEBUG [Listener at localhost/43647] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/.tmp/family2/718f9b0dd3454385a76ef2be7aeedd9f as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/718f9b0dd3454385a76ef2be7aeedd9f 2023-11-02 03:56:04,387 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 718f9b0dd3454385a76ef2be7aeedd9f 2023-11-02 03:56:04,388 INFO [Listener at localhost/43647] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 8d228d83c8f828dbf92eb31efd23c94f/family2 of 8d228d83c8f828dbf92eb31efd23c94f into 718f9b0dd3454385a76ef2be7aeedd9f(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-02 03:56:04,388 DEBUG [Listener at localhost/43647] regionserver.HRegion(2289): Compaction status journal for 8d228d83c8f828dbf92eb31efd23c94f: 2023-11-02 03:56:04,388 DEBUG [Listener at localhost/43647] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-02 03:56:04,389 DEBUG [Listener at localhost/43647] regionserver.HStore(1912): 6aad23f6dcc1d84b230e187242bb5c22/family1 is initiating major compaction (all files) 2023-11-02 03:56:04,389 INFO [Listener at localhost/43647] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-02 03:56:04,389 INFO [Listener at localhost/43647] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-02 03:56:04,390 INFO [Listener at localhost/43647] regionserver.HRegion(2259): Starting compaction of 6aad23f6dcc1d84b230e187242bb5c22/family1 in testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:56:04,390 INFO [Listener at localhost/43647] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/f88ce28b5fc748ae9f2eb61a46fbef4a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/60bd69509ca34cdf8864e80ef8d4079e, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/04d99b769dd346bf92c8c373d0bb3a38, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/a67948688f6343f1931d34b9bdbd4a80, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/90ce131dc98c4f47ada5908c9200fca9] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp, totalSize=27.1 K 2023-11-02 03:56:04,390 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting f88ce28b5fc748ae9f2eb61a46fbef4a, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897350989 2023-11-02 03:56:04,391 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 60bd69509ca34cdf8864e80ef8d4079e, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897352478 2023-11-02 03:56:04,391 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 04d99b769dd346bf92c8c373d0bb3a38, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897354347 2023-11-02 03:56:04,391 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting a67948688f6343f1931d34b9bdbd4a80, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897355827 2023-11-02 03:56:04,392 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 90ce131dc98c4f47ada5908c9200fca9, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-02 03:56:04,400 INFO [Listener at localhost/43647] throttle.PressureAwareThroughputController(145): 6aad23f6dcc1d84b230e187242bb5c22#family1#compaction#201 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:56:04,404 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/f354c3fef34c4b2f97ba59c918c09d8f, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/b440a8487a27436ba7e36c905e49d51a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/8bc5fc2d9772487eacbee60e48b3e715, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/10fe9b6d1e004888807871e9928a1755, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/f3f2ff453221472ab096046fb913b382] to archive 2023-11-02 03:56:04,404 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:56:04,405 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/f354c3fef34c4b2f97ba59c918c09d8f to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/f354c3fef34c4b2f97ba59c918c09d8f 2023-11-02 03:56:04,406 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/b440a8487a27436ba7e36c905e49d51a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/b440a8487a27436ba7e36c905e49d51a 2023-11-02 03:56:04,407 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/8bc5fc2d9772487eacbee60e48b3e715 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/8bc5fc2d9772487eacbee60e48b3e715 2023-11-02 03:56:04,408 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/10fe9b6d1e004888807871e9928a1755 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/10fe9b6d1e004888807871e9928a1755 2023-11-02 03:56:04,409 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/f3f2ff453221472ab096046fb913b382 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family2/f3f2ff453221472ab096046fb913b382 2023-11-02 03:56:04,844 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231102205514529da14db6af883898446d769f_del 2023-11-02 03:56:04,845 INFO [Listener at localhost/43647] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20231102205514529da14db6af883898446d769f_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231102205514529da14db6af883898446d769f_del 2023-11-02 03:56:05,705 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dff3ed39d41843d4af8caf7c9a8db02f 2023-11-02 03:56:05,706 DEBUG [Listener at localhost/43647] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family1/dff3ed39d41843d4af8caf7c9a8db02f as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/dff3ed39d41843d4af8caf7c9a8db02f 2023-11-02 03:56:05,727 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dff3ed39d41843d4af8caf7c9a8db02f 2023-11-02 03:56:05,728 INFO [Listener at localhost/43647] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6aad23f6dcc1d84b230e187242bb5c22/family1 of 6aad23f6dcc1d84b230e187242bb5c22 into dff3ed39d41843d4af8caf7c9a8db02f(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-02 03:56:05,728 DEBUG [Listener at localhost/43647] regionserver.HRegion(2289): Compaction status journal for 6aad23f6dcc1d84b230e187242bb5c22: 2023-11-02 03:56:05,728 DEBUG [Listener at localhost/43647] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-11-02 03:56:05,728 DEBUG [Listener at localhost/43647] regionserver.HStore(1912): 6aad23f6dcc1d84b230e187242bb5c22/family2 is initiating major compaction (all files) 2023-11-02 03:56:05,728 INFO [Listener at localhost/43647] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-11-02 03:56:05,728 INFO [Listener at localhost/43647] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-11-02 03:56:05,728 INFO [Listener at localhost/43647] regionserver.HRegion(2259): Starting compaction of 6aad23f6dcc1d84b230e187242bb5c22/family2 in testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:56:05,728 INFO [Listener at localhost/43647] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/b68ae350fea5466aa0f3674d88aef3c7, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/8c99bea1d05447e1b35dad5ebe16e3e6, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/a3a975da1da8484dbcda5812e11ad0f4, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/f08476af1202463fb61f227b1d1c42c1, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/f7dd9cbe0f5a4b14b5be5222dc9b0720] into tmpdir=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp, totalSize=25.4 K 2023-11-02 03:56:05,729 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting b68ae350fea5466aa0f3674d88aef3c7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1698897350989 2023-11-02 03:56:05,729 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting 8c99bea1d05447e1b35dad5ebe16e3e6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1698897352478 2023-11-02 03:56:05,729 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting a3a975da1da8484dbcda5812e11ad0f4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1698897354347 2023-11-02 03:56:05,730 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting f08476af1202463fb61f227b1d1c42c1, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1698897355827 2023-11-02 03:56:05,730 DEBUG [Listener at localhost/43647] compactions.Compactor(207): Compacting f7dd9cbe0f5a4b14b5be5222dc9b0720, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-11-02 03:56:05,736 INFO [Listener at localhost/43647] throttle.PressureAwareThroughputController(145): 6aad23f6dcc1d84b230e187242bb5c22#family2#compaction#202 average throughput is 1.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-11-02 03:56:05,804 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/f88ce28b5fc748ae9f2eb61a46fbef4a, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/60bd69509ca34cdf8864e80ef8d4079e, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/04d99b769dd346bf92c8c373d0bb3a38, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/a67948688f6343f1931d34b9bdbd4a80, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/90ce131dc98c4f47ada5908c9200fca9] to archive 2023-11-02 03:56:05,805 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:56:05,806 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/f88ce28b5fc748ae9f2eb61a46fbef4a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/f88ce28b5fc748ae9f2eb61a46fbef4a 2023-11-02 03:56:05,807 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/60bd69509ca34cdf8864e80ef8d4079e to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/60bd69509ca34cdf8864e80ef8d4079e 2023-11-02 03:56:05,808 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/04d99b769dd346bf92c8c373d0bb3a38 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/04d99b769dd346bf92c8c373d0bb3a38 2023-11-02 03:56:05,809 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/a67948688f6343f1931d34b9bdbd4a80 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/a67948688f6343f1931d34b9bdbd4a80 2023-11-02 03:56:05,810 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/90ce131dc98c4f47ada5908c9200fca9 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/90ce131dc98c4f47ada5908c9200fca9 2023-11-02 03:56:06,147 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202311024148fc4cda714501a769bba543c09e85_del 2023-11-02 03:56:06,147 INFO [Listener at localhost/43647] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202311024148fc4cda714501a769bba543c09e85_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202311024148fc4cda714501a769bba543c09e85_del 2023-11-02 03:56:06,960 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bbb1494fe0994a5798e25b6043986c01 2023-11-02 03:56:06,961 DEBUG [Listener at localhost/43647] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/.tmp/family2/bbb1494fe0994a5798e25b6043986c01 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/bbb1494fe0994a5798e25b6043986c01 2023-11-02 03:56:06,964 INFO [Listener at localhost/43647] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bbb1494fe0994a5798e25b6043986c01 2023-11-02 03:56:06,965 INFO [Listener at localhost/43647] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 6aad23f6dcc1d84b230e187242bb5c22/family2 of 6aad23f6dcc1d84b230e187242bb5c22 into bbb1494fe0994a5798e25b6043986c01(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-11-02 03:56:06,965 DEBUG [Listener at localhost/43647] regionserver.HRegion(2289): Compaction status journal for 6aad23f6dcc1d84b230e187242bb5c22: 2023-11-02 03:56:06,968 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-02 03:56:06,971 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-02 03:56:06,974 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-02 03:56:06,975 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-02 03:56:06,980 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41275] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10R\xBC\xA9\x93m4\x9FG"\x5CJXyR:\x87"\x10\xCB\xA0\xFE\x99IM\x0E\x9C\x13\xD73{\xAA\x9C\x1Ev*\x10\xF1;\x92#Mx\xFD\x0CnK\x85\x0C#\x80\xC6\xDE2\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-11-02 03:56:07,003 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/b68ae350fea5466aa0f3674d88aef3c7, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/8c99bea1d05447e1b35dad5ebe16e3e6, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/a3a975da1da8484dbcda5812e11ad0f4, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/f08476af1202463fb61f227b1d1c42c1, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/f7dd9cbe0f5a4b14b5be5222dc9b0720] to archive 2023-11-02 03:56:07,004 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:56:07,005 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/b68ae350fea5466aa0f3674d88aef3c7 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/b68ae350fea5466aa0f3674d88aef3c7 2023-11-02 03:56:07,006 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/8c99bea1d05447e1b35dad5ebe16e3e6 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/8c99bea1d05447e1b35dad5ebe16e3e6 2023-11-02 03:56:07,007 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/a3a975da1da8484dbcda5812e11ad0f4 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/a3a975da1da8484dbcda5812e11ad0f4 2023-11-02 03:56:07,008 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/f08476af1202463fb61f227b1d1c42c1 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/f08476af1202463fb61f227b1d1c42c1 2023-11-02 03:56:07,009 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/f7dd9cbe0f5a4b14b5be5222dc9b0720 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family2/f7dd9cbe0f5a4b14b5be5222dc9b0720 2023-11-02 03:56:07,013 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] procedure2.ProcedureExecutor(1028): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-11-02 03:56:07,014 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-11-02 03:56:07,014 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1698897967014 2023-11-02 03:56:07,014 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:56:07,015 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-11-02 03:56:07,343 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-11-02 03:56:07,364 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20231102205514529da14db6af883898446d769f_del 2023-11-02 03:56:07,386 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20231102f08d4fdde4cb44af8aca8064c6c8474f_del 2023-11-02 03:56:07,407 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202311023b9ce4666fb94b63812b53cc253133b5_del 2023-11-02 03:56:07,408 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-11-02 03:56:07,408 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] zookeeper.ReadOnlyZKClient(139): Connect 0x362ab71f to 127.0.0.1:65185 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-11-02 03:56:07,412 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3369ab34, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-11-02 03:56:07,413 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20231102 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-02 03:56:07,970 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20231102d5a98f72c74746c085b1d977f8a6c542 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231102d5a98f72c74746c085b1d977f8a6c542 2023-11-02 03:56:07,970 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231102 table='testMajorCompactionFromAdmin' column='family1' 2023-11-02 03:56:07,984 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-11-02 03:56:07,986 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40156, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-11-02 03:56:07,995 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-11-02 03:56:07,996 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47040, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-11-02 03:56:08,032 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231102/family1/99e7efc6e04448ffbb2461f15a239554 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-11-02 03:56:08,033 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231102/family1/99e7efc6e04448ffbb2461f15a239554}] 2023-11-02 03:56:08,037 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231102/family1/99e7efc6e04448ffbb2461f15a239554 for inclusion in 93842bec7617596d83d1e604c6d46170/family1 2023-11-02 03:56:08,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-11-02 03:56:08,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first=2 last= 2023-11-02 03:56:08,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for 93842bec7617596d83d1e604c6d46170: 2023-11-02 03:56:08,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231102/family1/99e7efc6e04448ffbb2461f15a239554 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionFromAdmin__sl5h8a444pnocu51e06h5v9jja04um8fcb5opnmn4da7c4g8qhb1r3k40gfnn4vd/family1/99e7efc6e04448ffbb2461f15a239554 2023-11-02 03:56:08,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionFromAdmin__sl5h8a444pnocu51e06h5v9jja04um8fcb5opnmn4da7c4g8qhb1r3k40gfnn4vd/family1/99e7efc6e04448ffbb2461f15a239554 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/7bfa407dd4664877a03d1cce790b909c_SeqId_28_ 2023-11-02 03:56:08,072 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionFromAdmin__sl5h8a444pnocu51e06h5v9jja04um8fcb5opnmn4da7c4g8qhb1r3k40gfnn4vd/family1/99e7efc6e04448ffbb2461f15a239554 into 93842bec7617596d83d1e604c6d46170/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/7bfa407dd4664877a03d1cce790b909c_SeqId_28_ - updating store file list. 2023-11-02 03:56:08,095 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/7bfa407dd4664877a03d1cce790b909c_SeqId_28_ into 93842bec7617596d83d1e604c6d46170/family1 2023-11-02 03:56:08,095 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionFromAdmin__sl5h8a444pnocu51e06h5v9jja04um8fcb5opnmn4da7c4g8qhb1r3k40gfnn4vd/family1/99e7efc6e04448ffbb2461f15a239554 into 93842bec7617596d83d1e604c6d46170/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/family1/7bfa407dd4664877a03d1cce790b909c_SeqId_28_) 2023-11-02 03:56:08,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionFromAdmin__sl5h8a444pnocu51e06h5v9jja04um8fcb5opnmn4da7c4g8qhb1r3k40gfnn4vd/family1/99e7efc6e04448ffbb2461f15a239554 2023-11-02 03:56:08,097 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20231102 table='testMajorCompactionFromAdmin' column='family1' 2023-11-02 03:56:08,098 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:56:08,099 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202311023065b46fc2004915a4429b04ea104c56 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202311023065b46fc2004915a4429b04ea104c56 2023-11-02 03:56:08,100 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023110267181fd1df1f4eba8f14f77c90574208 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023110267181fd1df1f4eba8f14f77c90574208 2023-11-02 03:56:08,102 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231102abe95fad387a463eb60e8b5c749efc33 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231102abe95fad387a463eb60e8b5c749efc33 2023-11-02 03:56:08,103 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231102e683f3fcd9b0479eb0875de05e4ce622 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20231102e683f3fcd9b0479eb0875de05e4ce622 2023-11-02 03:56:08,103 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-11-02 03:56:08,103 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20231102' 2023-11-02 03:56:08,103 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20231102 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-02 03:56:08,103 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20231102 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-02 03:56:08,657 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202311022582e48796d444f49e127e3153a3517d to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202311022582e48796d444f49e127e3153a3517d 2023-11-02 03:56:08,657 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231102 table='testMajorCompactionFromAdmin' column='family1' 2023-11-02 03:56:08,697 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231102/family1/e66ca5e2a95547979200052e9395642e first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-11-02 03:56:08,698 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231102/family1/e66ca5e2a95547979200052e9395642e}] 2023-11-02 03:56:08,700 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231102/family1/e66ca5e2a95547979200052e9395642e for inclusion in 8d228d83c8f828dbf92eb31efd23c94f/family1 2023-11-02 03:56:08,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-11-02 03:56:08,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first=1 last=2 2023-11-02 03:56:08,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for 8d228d83c8f828dbf92eb31efd23c94f: 2023-11-02 03:56:08,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231102/family1/e66ca5e2a95547979200052e9395642e to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionFromAdmin__ac79pbigpf8tetk4bkt5ir08svukp0iqk98rme98bgjmjhtn5r5linas8eh95m5c/family1/e66ca5e2a95547979200052e9395642e 2023-11-02 03:56:08,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionFromAdmin__ac79pbigpf8tetk4bkt5ir08svukp0iqk98rme98bgjmjhtn5r5linas8eh95m5c/family1/e66ca5e2a95547979200052e9395642e as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/9b9309e6c67d480ebe0d42eba3c67fb2_SeqId_28_ 2023-11-02 03:56:08,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionFromAdmin__ac79pbigpf8tetk4bkt5ir08svukp0iqk98rme98bgjmjhtn5r5linas8eh95m5c/family1/e66ca5e2a95547979200052e9395642e into 8d228d83c8f828dbf92eb31efd23c94f/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/9b9309e6c67d480ebe0d42eba3c67fb2_SeqId_28_ - updating store file list. 2023-11-02 03:56:08,743 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/9b9309e6c67d480ebe0d42eba3c67fb2_SeqId_28_ into 8d228d83c8f828dbf92eb31efd23c94f/family1 2023-11-02 03:56:08,743 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionFromAdmin__ac79pbigpf8tetk4bkt5ir08svukp0iqk98rme98bgjmjhtn5r5linas8eh95m5c/family1/e66ca5e2a95547979200052e9395642e into 8d228d83c8f828dbf92eb31efd23c94f/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/family1/9b9309e6c67d480ebe0d42eba3c67fb2_SeqId_28_) 2023-11-02 03:56:08,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionFromAdmin__ac79pbigpf8tetk4bkt5ir08svukp0iqk98rme98bgjmjhtn5r5linas8eh95m5c/family1/e66ca5e2a95547979200052e9395642e 2023-11-02 03:56:08,745 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20231102 table='testMajorCompactionFromAdmin' column='family1' 2023-11-02 03:56:08,746 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:56:08,747 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202311023f1b7c4592e44fa4ba2ac142dec48851 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202311023f1b7c4592e44fa4ba2ac142dec48851 2023-11-02 03:56:08,748 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231102a4bb97cc1c4b4b5abe66b65941226c95 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231102a4bb97cc1c4b4b5abe66b65941226c95 2023-11-02 03:56:08,749 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231102c2ebf8fd237d42b88c572dbc24e08482 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231102c2ebf8fd237d42b88c572dbc24e08482 2023-11-02 03:56:08,750 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231102dab4c893958d4cf8a0e61351c394891a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231102dab4c893958d4cf8a0e61351c394891a 2023-11-02 03:56:08,750 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-11-02 03:56:08,750 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20231102' 2023-11-02 03:56:08,750 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20231102 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-02 03:56:08,750 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20231102 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-02 03:56:09,718 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023110203288cbe25eb460db25fdc053f66e29a to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023110203288cbe25eb460db25fdc053f66e29a 2023-11-02 03:56:09,719 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231102 table='testMajorCompactionFromAdmin' column='family1' 2023-11-02 03:56:09,758 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(788): Trying to load hfile=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231102/family1/4d6bcdcdae0442d7a0d3ff4dabf4b7ab first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-11-02 03:56:09,759 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(553): Going to connect to server region=testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22., hostname=jenkins-hbase4.apache.org,36627,1698897201574, seqNum=2 for row with hfile group [{family1,hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231102/family1/4d6bcdcdae0442d7a0d3ff4dabf4b7ab}] 2023-11-02 03:56:09,761 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(763): Validating hfile at hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231102/family1/4d6bcdcdae0442d7a0d3ff4dabf4b7ab for inclusion in 6aad23f6dcc1d84b230e187242bb5c22/family1 2023-11-02 03:56:09,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-11-02 03:56:09,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(777): Region bounds: first= last=1 2023-11-02 03:56:09,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegion(2446): Flush status journal for 6aad23f6dcc1d84b230e187242bb5c22: 2023-11-02 03:56:09,782 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231102/family1/4d6bcdcdae0442d7a0d3ff4dabf4b7ab to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionFromAdmin__q6fhsuvu30eds1ip1h3927o69sns8o4e2esvdsrtaikppogsjfdruogca0aeles3/family1/4d6bcdcdae0442d7a0d3ff4dabf4b7ab 2023-11-02 03:56:09,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionFromAdmin__q6fhsuvu30eds1ip1h3927o69sns8o4e2esvdsrtaikppogsjfdruogca0aeles3/family1/4d6bcdcdae0442d7a0d3ff4dabf4b7ab as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/84a7c183d17447f58a18d1ee81770f3c_SeqId_28_ 2023-11-02 03:56:09,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(849): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionFromAdmin__q6fhsuvu30eds1ip1h3927o69sns8o4e2esvdsrtaikppogsjfdruogca0aeles3/family1/4d6bcdcdae0442d7a0d3ff4dabf4b7ab into 6aad23f6dcc1d84b230e187242bb5c22/family1 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/84a7c183d17447f58a18d1ee81770f3c_SeqId_28_ - updating store file list. 2023-11-02 03:56:09,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(882): Loaded HFile hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/84a7c183d17447f58a18d1ee81770f3c_SeqId_28_ into 6aad23f6dcc1d84b230e187242bb5c22/family1 2023-11-02 03:56:09,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.HStore(855): Successfully loaded hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionFromAdmin__q6fhsuvu30eds1ip1h3927o69sns8o4e2esvdsrtaikppogsjfdruogca0aeles3/family1/4d6bcdcdae0442d7a0d3ff4dabf4b7ab into 6aad23f6dcc1d84b230e187242bb5c22/family1 (new location: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/family1/84a7c183d17447f58a18d1ee81770f3c_SeqId_28_) 2023-11-02 03:56:09,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36627] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/staging/jenkins__testMajorCompactionFromAdmin__q6fhsuvu30eds1ip1h3927o69sns8o4e2esvdsrtaikppogsjfdruogca0aeles3/family1/4d6bcdcdae0442d7a0d3ff4dabf4b7ab 2023-11-02 03:56:09,807 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20231102 table='testMajorCompactionFromAdmin' column='family1' 2023-11-02 03:56:09,808 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:56:09,809 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202311020ee657e2472e4d6d87ba06147e0cb623 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202311020ee657e2472e4d6d87ba06147e0cb623 2023-11-02 03:56:09,810 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023110218aaed54c87b499887bc5a77cc2b2811 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023110218aaed54c87b499887bc5a77cc2b2811 2023-11-02 03:56:09,810 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231102a8bf95ca22744b95884535fa59314a93 to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231102a8bf95ca22744b95884535fa59314a93 2023-11-02 03:56:09,811 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231102ffbb32101862408389f4da0beba804ff to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231102ffbb32101862408389f4da0beba804ff 2023-11-02 03:56:09,811 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-11-02 03:56:09,811 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20231102' 2023-11-02 03:56:09,812 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20231102 for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-02 03:56:09,812 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-11-02 03:56:09,812 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-11-02 03:56:09,812 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(367): [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231102205514529da14db6af883898446d769f_del] 2023-11-02 03:56:09,812 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:56:09,814 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231102205514529da14db6af883898446d769f_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20231102205514529da14db6af883898446d769f_del 2023-11-02 03:56:09,814 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-11-02 03:56:09,814 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(367): [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231102f08d4fdde4cb44af8aca8064c6c8474f_del] 2023-11-02 03:56:09,814 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:56:09,815 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231102f08d4fdde4cb44af8aca8064c6c8474f_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20231102f08d4fdde4cb44af8aca8064c6c8474f_del 2023-11-02 03:56:09,815 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-11-02 03:56:09,816 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] compactions.PartitionedMobCompactor(367): [hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202311023b9ce4666fb94b63812b53cc253133b5_del] 2023-11-02 03:56:09,816 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] backup.HFileArchiver(360): Archiving compacted files. 2023-11-02 03:56:09,817 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202311023b9ce4666fb94b63812b53cc253133b5_del to hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202311023b9ce4666fb94b63812b53cc253133b5_del 2023-11-02 03:56:09,817 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-11-02 03:56:09,817 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1698897204678] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-11-02 03:56:09,818 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-11-02 03:56:09,819 INFO [PEWorker-3] procedure2.ProcedureExecutor(1409): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 2.8070 sec 2023-11-02 03:56:09,836 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-02 03:56:09,838 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-02 03:56:09,840 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-11-02 03:56:09,841 DEBUG [Listener at localhost/43647] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-11-02 03:56:09,845 INFO [Listener at localhost/43647] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-11-02 03:56:09,880 INFO [Listener at localhost/43647] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=250 (was 263), OpenFileDescriptor=659 (was 686), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=115 (was 112) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 167) - ProcessCount LEAK? -, AvailableMemoryMB=11457 (was 11476) 2023-11-02 03:56:09,881 DEBUG [Listener at localhost/43647] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x75a9267a to 127.0.0.1:65185 2023-11-02 03:56:09,881 DEBUG [Listener at localhost/43647] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:56:09,881 INFO [Listener at localhost/43647] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-11-02 03:56:09,881 INFO [Listener at localhost/43647] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-11-02 03:56:09,881 DEBUG [Listener at localhost/43647] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0e51ea3b to 127.0.0.1:65185 2023-11-02 03:56:09,881 DEBUG [Listener at localhost/43647] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:56:09,881 DEBUG [Listener at localhost/43647] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-11-02 03:56:09,882 DEBUG [Listener at localhost/43647] util.JVMClusterUtil(257): Found active master hash=229085947, stopped=false 2023-11-02 03:56:09,882 INFO [Listener at localhost/43647] master.ServerManager(888): Cluster shutdown requested of master=jenkins-hbase4.apache.org,41275,1698897200405 2023-11-02 03:56:09,883 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-11-02 03:56:09,883 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-11-02 03:56:09,883 INFO [Listener at localhost/43647] procedure2.ProcedureExecutor(628): Stopping 2023-11-02 03:56:09,883 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-02 03:56:09,884 DEBUG [Listener at localhost/43647] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x73f47c81 to 127.0.0.1:65185 2023-11-02 03:56:09,884 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-02 03:56:09,884 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-11-02 03:56:09,884 DEBUG [Listener at localhost/43647] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:56:09,884 INFO [Listener at localhost/43647] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,36627,1698897201574' ***** 2023-11-02 03:56:09,884 INFO [Listener at localhost/43647] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-11-02 03:56:09,884 INFO [RS:0;jenkins-hbase4:36627] regionserver.HeapMemoryManager(220): Stopping 2023-11-02 03:56:09,885 INFO [RS:0;jenkins-hbase4:36627] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-11-02 03:56:09,885 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-11-02 03:56:09,885 INFO [RS:0;jenkins-hbase4:36627] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-11-02 03:56:09,885 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(3305): Received CLOSE for 93842bec7617596d83d1e604c6d46170 2023-11-02 03:56:09,886 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(3305): Received CLOSE for cb0f3da5a940148bb91c8b121c586d73 2023-11-02 03:56:09,886 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(3305): Received CLOSE for c21eb15d0957cd8a2c44a1d684d8b7de 2023-11-02 03:56:09,886 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 93842bec7617596d83d1e604c6d46170, disabling compactions & flushes 2023-11-02 03:56:09,886 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(3305): Received CLOSE for e9d73087146f1976779ec867087e57e1 2023-11-02 03:56:09,886 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:56:09,886 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(3305): Received CLOSE for d2c7ce048f92242cd7278c0667f09a6d 2023-11-02 03:56:09,886 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:56:09,886 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(3305): Received CLOSE for 2b1e8f467362f169103ba63e90e10342 2023-11-02 03:56:09,886 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. after waiting 0 ms 2023-11-02 03:56:09,886 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(3305): Received CLOSE for 028924ec081a2d5ff15a43f486a54e39 2023-11-02 03:56:09,886 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:56:09,886 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(3305): Received CLOSE for dd77f6ddad975771f984fe41bad1dd90 2023-11-02 03:56:09,886 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(3305): Received CLOSE for fadbd61541ad75cfef60a4b6222d7f6b 2023-11-02 03:56:09,886 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(3305): Received CLOSE for 8f6ce6f475709f0e3737fe28a4b8cba4 2023-11-02 03:56:09,886 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(3305): Received CLOSE for 8137557c26d83576967aea0c8ee5cd27 2023-11-02 03:56:09,886 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(3305): Received CLOSE for 6ec4530a62d461a9c6949db814d37fd3 2023-11-02 03:56:09,886 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(3305): Received CLOSE for 8d228d83c8f828dbf92eb31efd23c94f 2023-11-02 03:56:09,886 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(3305): Received CLOSE for 9a8949a1c626d50be08da97ed0a2c69e 2023-11-02 03:56:09,886 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(3305): Received CLOSE for 6aad23f6dcc1d84b230e187242bb5c22 2023-11-02 03:56:09,886 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(3305): Received CLOSE for bba4c1e5e128c37e7ef0161318ba5c99 2023-11-02 03:56:09,886 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(3305): Received CLOSE for 2d76cfa810f9f3428afc40975d277f1d 2023-11-02 03:56:09,886 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(3305): Received CLOSE for c9354336ff3eea83ac372fc04a8d438e 2023-11-02 03:56:09,886 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(3305): Received CLOSE for 06d225f5eeba431dae3475159f87d35e 2023-11-02 03:56:09,887 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(3305): Received CLOSE for b478efa9db26260ecef7476a80a8478c 2023-11-02 03:56:09,887 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:56:09,887 DEBUG [RS:0;jenkins-hbase4:36627] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x320de6d4 to 127.0.0.1:65185 2023-11-02 03:56:09,887 DEBUG [RS:0;jenkins-hbase4:36627] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:56:09,887 INFO [RS:0;jenkins-hbase4:36627] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-11-02 03:56:09,887 INFO [RS:0;jenkins-hbase4:36627] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-11-02 03:56:09,887 INFO [RS:0;jenkins-hbase4:36627] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-11-02 03:56:09,887 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-11-02 03:56:09,888 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(1474): Waiting on 21 regions to close 2023-11-02 03:56:09,888 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-11-02 03:56:09,888 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-11-02 03:56:09,888 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-11-02 03:56:09,888 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-11-02 03:56:09,888 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-11-02 03:56:09,888 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-11-02 03:56:09,888 DEBUG [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(1504): Waiting on 028924ec081a2d5ff15a43f486a54e39, 06d225f5eeba431dae3475159f87d35e, 1588230740, 2b1e8f467362f169103ba63e90e10342, 2d76cfa810f9f3428afc40975d277f1d, 6aad23f6dcc1d84b230e187242bb5c22, 6ec4530a62d461a9c6949db814d37fd3, 8137557c26d83576967aea0c8ee5cd27, 8d228d83c8f828dbf92eb31efd23c94f, 8f6ce6f475709f0e3737fe28a4b8cba4, 93842bec7617596d83d1e604c6d46170, 9a8949a1c626d50be08da97ed0a2c69e, b478efa9db26260ecef7476a80a8478c, bba4c1e5e128c37e7ef0161318ba5c99, c21eb15d0957cd8a2c44a1d684d8b7de, c9354336ff3eea83ac372fc04a8d438e, cb0f3da5a940148bb91c8b121c586d73, d2c7ce048f92242cd7278c0667f09a6d, dd77f6ddad975771f984fe41bad1dd90, e9d73087146f1976779ec867087e57e1, fadbd61541ad75cfef60a4b6222d7f6b 2023-11-02 03:56:09,890 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/93842bec7617596d83d1e604c6d46170/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-02 03:56:09,891 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:56:09,891 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 93842bec7617596d83d1e604c6d46170: 2023-11-02 03:56:09,891 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1698897348845.93842bec7617596d83d1e604c6d46170. 2023-11-02 03:56:09,892 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing cb0f3da5a940148bb91c8b121c586d73, disabling compactions & flushes 2023-11-02 03:56:09,892 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:56:09,892 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:56:09,892 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. after waiting 0 ms 2023-11-02 03:56:09,892 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:56:09,898 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyPolicy/cb0f3da5a940148bb91c8b121c586d73/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-11-02 03:56:09,899 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:56:09,899 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for cb0f3da5a940148bb91c8b121c586d73: 2023-11-02 03:56:09,899 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1698897244578.cb0f3da5a940148bb91c8b121c586d73. 2023-11-02 03:56:09,899 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c21eb15d0957cd8a2c44a1d684d8b7de, disabling compactions & flushes 2023-11-02 03:56:09,899 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:56:09,899 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:56:09,899 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. after waiting 0 ms 2023-11-02 03:56:09,899 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:56:09,900 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/.tmp/info/cd5fa72b22e34b82a1c9844f47f3fd62 2023-11-02 03:56:09,902 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithMonthlyPolicy/c21eb15d0957cd8a2c44a1d684d8b7de/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-11-02 03:56:09,903 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:56:09,903 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c21eb15d0957cd8a2c44a1d684d8b7de: 2023-11-02 03:56:09,903 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1698897216818.c21eb15d0957cd8a2c44a1d684d8b7de. 2023-11-02 03:56:09,903 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e9d73087146f1976779ec867087e57e1, disabling compactions & flushes 2023-11-02 03:56:09,903 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1. 2023-11-02 03:56:09,903 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1. 2023-11-02 03:56:09,903 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1. after waiting 0 ms 2023-11-02 03:56:09,903 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1. 2023-11-02 03:56:09,903 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing e9d73087146f1976779ec867087e57e1 1/1 column families, dataSize=109 B heapSize=592 B 2023-11-02 03:56:09,904 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cd5fa72b22e34b82a1c9844f47f3fd62 2023-11-02 03:56:09,912 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-11-02 03:56:09,953 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-11-02 03:56:09,953 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-11-02 03:56:09,962 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-11-02 03:56:10,089 DEBUG [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(1504): Waiting on 028924ec081a2d5ff15a43f486a54e39, 06d225f5eeba431dae3475159f87d35e, 1588230740, 2b1e8f467362f169103ba63e90e10342, 2d76cfa810f9f3428afc40975d277f1d, 6aad23f6dcc1d84b230e187242bb5c22, 6ec4530a62d461a9c6949db814d37fd3, 8137557c26d83576967aea0c8ee5cd27, 8d228d83c8f828dbf92eb31efd23c94f, 8f6ce6f475709f0e3737fe28a4b8cba4, 9a8949a1c626d50be08da97ed0a2c69e, b478efa9db26260ecef7476a80a8478c, bba4c1e5e128c37e7ef0161318ba5c99, c9354336ff3eea83ac372fc04a8d438e, d2c7ce048f92242cd7278c0667f09a6d, dd77f6ddad975771f984fe41bad1dd90, e9d73087146f1976779ec867087e57e1, fadbd61541ad75cfef60a4b6222d7f6b 2023-11-02 03:56:10,289 DEBUG [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(1504): Waiting on 028924ec081a2d5ff15a43f486a54e39, 06d225f5eeba431dae3475159f87d35e, 1588230740, 2b1e8f467362f169103ba63e90e10342, 2d76cfa810f9f3428afc40975d277f1d, 6aad23f6dcc1d84b230e187242bb5c22, 6ec4530a62d461a9c6949db814d37fd3, 8137557c26d83576967aea0c8ee5cd27, 8d228d83c8f828dbf92eb31efd23c94f, 8f6ce6f475709f0e3737fe28a4b8cba4, 9a8949a1c626d50be08da97ed0a2c69e, b478efa9db26260ecef7476a80a8478c, bba4c1e5e128c37e7ef0161318ba5c99, c9354336ff3eea83ac372fc04a8d438e, d2c7ce048f92242cd7278c0667f09a6d, dd77f6ddad975771f984fe41bad1dd90, e9d73087146f1976779ec867087e57e1, fadbd61541ad75cfef60a4b6222d7f6b 2023-11-02 03:56:10,311 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/namespace/e9d73087146f1976779ec867087e57e1/.tmp/info/7dae6e40ba1a441daade9eee7a8dafa7 2023-11-02 03:56:10,312 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/.tmp/rep_barrier/a0e6c761a81842369eb5ec80aa0ac1a3 2023-11-02 03:56:10,315 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/namespace/e9d73087146f1976779ec867087e57e1/.tmp/info/7dae6e40ba1a441daade9eee7a8dafa7 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/namespace/e9d73087146f1976779ec867087e57e1/info/7dae6e40ba1a441daade9eee7a8dafa7 2023-11-02 03:56:10,315 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a0e6c761a81842369eb5ec80aa0ac1a3 2023-11-02 03:56:10,319 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/namespace/e9d73087146f1976779ec867087e57e1/info/7dae6e40ba1a441daade9eee7a8dafa7, entries=3, sequenceid=7, filesize=4.8 K 2023-11-02 03:56:10,320 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for e9d73087146f1976779ec867087e57e1 in 417ms, sequenceid=7, compaction requested=false 2023-11-02 03:56:10,325 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/namespace/e9d73087146f1976779ec867087e57e1/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-11-02 03:56:10,326 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1. 2023-11-02 03:56:10,326 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/.tmp/table/07a2e5af2d594136bf3ee57884cfe408 2023-11-02 03:56:10,326 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e9d73087146f1976779ec867087e57e1: 2023-11-02 03:56:10,326 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1698897203548.e9d73087146f1976779ec867087e57e1. 2023-11-02 03:56:10,326 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d2c7ce048f92242cd7278c0667f09a6d, disabling compactions & flushes 2023-11-02 03:56:10,326 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:56:10,326 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:56:10,327 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. after waiting 0 ms 2023-11-02 03:56:10,327 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:56:10,330 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 07a2e5af2d594136bf3ee57884cfe408 2023-11-02 03:56:10,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithWeeklyPolicy/d2c7ce048f92242cd7278c0667f09a6d/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-11-02 03:56:10,331 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:56:10,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d2c7ce048f92242cd7278c0667f09a6d: 2023-11-02 03:56:10,331 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/.tmp/info/cd5fa72b22e34b82a1c9844f47f3fd62 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/info/cd5fa72b22e34b82a1c9844f47f3fd62 2023-11-02 03:56:10,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1698897229048.d2c7ce048f92242cd7278c0667f09a6d. 2023-11-02 03:56:10,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2b1e8f467362f169103ba63e90e10342, disabling compactions & flushes 2023-11-02 03:56:10,331 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:56:10,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:56:10,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. after waiting 0 ms 2023-11-02 03:56:10,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:56:10,334 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2b1e8f467362f169103ba63e90e10342/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-02 03:56:10,334 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cd5fa72b22e34b82a1c9844f47f3fd62 2023-11-02 03:56:10,334 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/info/cd5fa72b22e34b82a1c9844f47f3fd62, entries=226, sequenceid=130, filesize=37.9 K 2023-11-02 03:56:10,335 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:56:10,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2b1e8f467362f169103ba63e90e10342: 2023-11-02 03:56:10,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1698897321385.2b1e8f467362f169103ba63e90e10342. 2023-11-02 03:56:10,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 028924ec081a2d5ff15a43f486a54e39, disabling compactions & flushes 2023-11-02 03:56:10,335 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:56:10,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:56:10,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. after waiting 0 ms 2023-11-02 03:56:10,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:56:10,336 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/.tmp/rep_barrier/a0e6c761a81842369eb5ec80aa0ac1a3 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/rep_barrier/a0e6c761a81842369eb5ec80aa0ac1a3 2023-11-02 03:56:10,338 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMinorCompactionWithMonthlyPolicy/028924ec081a2d5ff15a43f486a54e39/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-11-02 03:56:10,339 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:56:10,340 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 028924ec081a2d5ff15a43f486a54e39: 2023-11-02 03:56:10,340 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1698897337379.028924ec081a2d5ff15a43f486a54e39. 2023-11-02 03:56:10,340 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing dd77f6ddad975771f984fe41bad1dd90, disabling compactions & flushes 2023-11-02 03:56:10,340 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:56:10,340 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:56:10,340 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a0e6c761a81842369eb5ec80aa0ac1a3 2023-11-02 03:56:10,340 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. after waiting 0 ms 2023-11-02 03:56:10,340 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/rep_barrier/a0e6c761a81842369eb5ec80aa0ac1a3, entries=1, sequenceid=130, filesize=5.0 K 2023-11-02 03:56:10,340 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:56:10,341 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/.tmp/table/07a2e5af2d594136bf3ee57884cfe408 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/table/07a2e5af2d594136bf3ee57884cfe408 2023-11-02 03:56:10,345 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 07a2e5af2d594136bf3ee57884cfe408 2023-11-02 03:56:10,345 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/table/07a2e5af2d594136bf3ee57884cfe408, entries=27, sequenceid=130, filesize=6.6 K 2023-11-02 03:56:10,347 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 459ms, sequenceid=130, compaction requested=false 2023-11-02 03:56:10,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/dd77f6ddad975771f984fe41bad1dd90/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-11-02 03:56:10,350 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:56:10,350 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for dd77f6ddad975771f984fe41bad1dd90: 2023-11-02 03:56:10,350 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1698897295808.dd77f6ddad975771f984fe41bad1dd90. 2023-11-02 03:56:10,350 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing fadbd61541ad75cfef60a4b6222d7f6b, disabling compactions & flushes 2023-11-02 03:56:10,350 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b. 2023-11-02 03:56:10,350 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b. 2023-11-02 03:56:10,350 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b. after waiting 0 ms 2023-11-02 03:56:10,350 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b. 2023-11-02 03:56:10,355 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-11-02 03:56:10,355 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-11-02 03:56:10,356 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-11-02 03:56:10,356 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-11-02 03:56:10,356 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-11-02 03:56:10,358 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/fadbd61541ad75cfef60a4b6222d7f6b/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-11-02 03:56:10,359 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b. 2023-11-02 03:56:10,359 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for fadbd61541ad75cfef60a4b6222d7f6b: 2023-11-02 03:56:10,359 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1698897241291.fadbd61541ad75cfef60a4b6222d7f6b. 2023-11-02 03:56:10,359 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8f6ce6f475709f0e3737fe28a4b8cba4, disabling compactions & flushes 2023-11-02 03:56:10,359 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:56:10,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:56:10,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. after waiting 0 ms 2023-11-02 03:56:10,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:56:10,362 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/8f6ce6f475709f0e3737fe28a4b8cba4/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-11-02 03:56:10,363 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:56:10,363 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8f6ce6f475709f0e3737fe28a4b8cba4: 2023-11-02 03:56:10,363 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1698897204816.8f6ce6f475709f0e3737fe28a4b8cba4. 2023-11-02 03:56:10,363 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8137557c26d83576967aea0c8ee5cd27, disabling compactions & flushes 2023-11-02 03:56:10,364 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27. 2023-11-02 03:56:10,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27. 2023-11-02 03:56:10,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27. after waiting 0 ms 2023-11-02 03:56:10,364 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27. 2023-11-02 03:56:10,366 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/8137557c26d83576967aea0c8ee5cd27/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-11-02 03:56:10,367 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27. 2023-11-02 03:56:10,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8137557c26d83576967aea0c8ee5cd27: 2023-11-02 03:56:10,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1698897241291.8137557c26d83576967aea0c8ee5cd27. 2023-11-02 03:56:10,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6ec4530a62d461a9c6949db814d37fd3, disabling compactions & flushes 2023-11-02 03:56:10,367 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:56:10,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:56:10,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. after waiting 0 ms 2023-11-02 03:56:10,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:56:10,370 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/6ec4530a62d461a9c6949db814d37fd3/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-11-02 03:56:10,371 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:56:10,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6ec4530a62d461a9c6949db814d37fd3: 2023-11-02 03:56:10,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1698897295808.6ec4530a62d461a9c6949db814d37fd3. 2023-11-02 03:56:10,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8d228d83c8f828dbf92eb31efd23c94f, disabling compactions & flushes 2023-11-02 03:56:10,371 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:56:10,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:56:10,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. after waiting 0 ms 2023-11-02 03:56:10,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:56:10,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/8d228d83c8f828dbf92eb31efd23c94f/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-02 03:56:10,374 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:56:10,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8d228d83c8f828dbf92eb31efd23c94f: 2023-11-02 03:56:10,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1698897348845.8d228d83c8f828dbf92eb31efd23c94f. 2023-11-02 03:56:10,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 9a8949a1c626d50be08da97ed0a2c69e, disabling compactions & flushes 2023-11-02 03:56:10,375 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:56:10,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:56:10,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. after waiting 0 ms 2023-11-02 03:56:10,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:56:10,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/9a8949a1c626d50be08da97ed0a2c69e/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-02 03:56:10,378 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:56:10,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 9a8949a1c626d50be08da97ed0a2c69e: 2023-11-02 03:56:10,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1698897321385.9a8949a1c626d50be08da97ed0a2c69e. 2023-11-02 03:56:10,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6aad23f6dcc1d84b230e187242bb5c22, disabling compactions & flushes 2023-11-02 03:56:10,378 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:56:10,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:56:10,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. after waiting 0 ms 2023-11-02 03:56:10,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:56:10,382 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionFromAdmin/6aad23f6dcc1d84b230e187242bb5c22/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-02 03:56:10,383 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:56:10,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6aad23f6dcc1d84b230e187242bb5c22: 2023-11-02 03:56:10,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1698897348845.6aad23f6dcc1d84b230e187242bb5c22. 2023-11-02 03:56:10,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing bba4c1e5e128c37e7ef0161318ba5c99, disabling compactions & flushes 2023-11-02 03:56:10,383 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99. 2023-11-02 03:56:10,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99. 2023-11-02 03:56:10,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99. after waiting 0 ms 2023-11-02 03:56:10,383 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99. 2023-11-02 03:56:10,385 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testScannerOnBulkLoadRefHFiles/bba4c1e5e128c37e7ef0161318ba5c99/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-11-02 03:56:10,386 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99. 2023-11-02 03:56:10,386 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for bba4c1e5e128c37e7ef0161318ba5c99: 2023-11-02 03:56:10,386 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1698897241291.bba4c1e5e128c37e7ef0161318ba5c99. 2023-11-02 03:56:10,386 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2d76cfa810f9f3428afc40975d277f1d, disabling compactions & flushes 2023-11-02 03:56:10,386 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:56:10,386 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:56:10,386 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. after waiting 0 ms 2023-11-02 03:56:10,386 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:56:10,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/ns/testMinorCompaction/2d76cfa810f9f3428afc40975d277f1d/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-11-02 03:56:10,389 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:56:10,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2d76cfa810f9f3428afc40975d277f1d: 2023-11-02 03:56:10,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1698897321385.2d76cfa810f9f3428afc40975d277f1d. 2023-11-02 03:56:10,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c9354336ff3eea83ac372fc04a8d438e, disabling compactions & flushes 2023-11-02 03:56:10,389 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:56:10,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:56:10,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. after waiting 0 ms 2023-11-02 03:56:10,389 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:56:10,392 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/c9354336ff3eea83ac372fc04a8d438e/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-11-02 03:56:10,392 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:56:10,392 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c9354336ff3eea83ac372fc04a8d438e: 2023-11-02 03:56:10,392 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1698897273253.c9354336ff3eea83ac372fc04a8d438e. 2023-11-02 03:56:10,392 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 06d225f5eeba431dae3475159f87d35e, disabling compactions & flushes 2023-11-02 03:56:10,393 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:56:10,393 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:56:10,393 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. after waiting 0 ms 2023-11-02 03:56:10,393 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:56:10,395 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/06d225f5eeba431dae3475159f87d35e/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-11-02 03:56:10,395 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:56:10,395 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 06d225f5eeba431dae3475159f87d35e: 2023-11-02 03:56:10,395 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1698897257452.06d225f5eeba431dae3475159f87d35e. 2023-11-02 03:56:10,395 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b478efa9db26260ecef7476a80a8478c, disabling compactions & flushes 2023-11-02 03:56:10,395 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:56:10,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:56:10,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. after waiting 0 ms 2023-11-02 03:56:10,396 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:56:10,398 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/data/default/testCompactionWithHFileLink/b478efa9db26260ecef7476a80a8478c/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-11-02 03:56:10,398 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:56:10,398 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b478efa9db26260ecef7476a80a8478c: 2023-11-02 03:56:10,398 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1698897295808.b478efa9db26260ecef7476a80a8478c. 2023-11-02 03:56:10,489 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,36627,1698897201574; all regions closed. 2023-11-02 03:56:10,498 DEBUG [RS:0;jenkins-hbase4:36627] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/oldWALs 2023-11-02 03:56:10,498 INFO [RS:0;jenkins-hbase4:36627] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36627%2C1698897201574.meta:.meta(num 1698897203289) 2023-11-02 03:56:10,502 DEBUG [RS:0;jenkins-hbase4:36627] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/oldWALs 2023-11-02 03:56:10,502 INFO [RS:0;jenkins-hbase4:36627] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36627%2C1698897201574:(num 1698897203108) 2023-11-02 03:56:10,502 DEBUG [RS:0;jenkins-hbase4:36627] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:56:10,502 INFO [RS:0;jenkins-hbase4:36627] regionserver.LeaseManager(133): Closed leases 2023-11-02 03:56:10,502 INFO [RS:0;jenkins-hbase4:36627] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-11-02 03:56:10,503 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-11-02 03:56:10,504 INFO [RS:0;jenkins-hbase4:36627] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36627 2023-11-02 03:56:10,513 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-11-02 03:56:10,513 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36627,1698897201574 2023-11-02 03:56:10,513 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-11-02 03:56:10,514 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,36627,1698897201574] 2023-11-02 03:56:10,515 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36627,1698897201574; numProcessing=1 2023-11-02 03:56:10,516 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,36627,1698897201574 already deleted, retry=false 2023-11-02 03:56:10,516 INFO [RegionServerTracker-0] master.ServerManager(554): Cluster shutdown set; jenkins-hbase4.apache.org,36627,1698897201574 expired; onlineServers=0 2023-11-02 03:56:10,516 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,41275,1698897200405' ***** 2023-11-02 03:56:10,516 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-11-02 03:56:10,516 DEBUG [M:0;jenkins-hbase4:41275] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4c9a98c0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-11-02 03:56:10,516 INFO [M:0;jenkins-hbase4:41275] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,41275,1698897200405 2023-11-02 03:56:10,517 INFO [M:0;jenkins-hbase4:41275] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,41275,1698897200405; all regions closed. 2023-11-02 03:56:10,517 DEBUG [M:0;jenkins-hbase4:41275] ipc.AbstractRpcClient(494): Stopping rpc client 2023-11-02 03:56:10,517 DEBUG [M:0;jenkins-hbase4:41275] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-11-02 03:56:10,517 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-11-02 03:56:10,517 DEBUG [M:0;jenkins-hbase4:41275] cleaner.HFileCleaner(317): Stopping file delete threads 2023-11-02 03:56:10,517 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1698897202813] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1698897202813,5,FailOnTimeoutGroup] 2023-11-02 03:56:10,517 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1698897202814] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1698897202814,5,FailOnTimeoutGroup] 2023-11-02 03:56:10,517 INFO [M:0;jenkins-hbase4:41275] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-11-02 03:56:10,517 INFO [M:0;jenkins-hbase4:41275] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-11-02 03:56:10,517 INFO [M:0;jenkins-hbase4:41275] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-11-02 03:56:10,518 DEBUG [M:0;jenkins-hbase4:41275] master.HMaster(1512): Stopping service threads 2023-11-02 03:56:10,518 INFO [M:0;jenkins-hbase4:41275] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-11-02 03:56:10,518 ERROR [M:0;jenkins-hbase4:41275] procedure2.ProcedureExecutor(652): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-11-02 03:56:10,519 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-11-02 03:56:10,519 INFO [M:0;jenkins-hbase4:41275] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-11-02 03:56:10,519 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-11-02 03:56:10,519 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-11-02 03:56:10,519 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-11-02 03:56:10,519 DEBUG [M:0;jenkins-hbase4:41275] zookeeper.ZKUtil(398): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-11-02 03:56:10,519 WARN [M:0;jenkins-hbase4:41275] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-11-02 03:56:10,520 INFO [M:0;jenkins-hbase4:41275] assignment.AssignmentManager(315): Stopping assignment manager 2023-11-02 03:56:10,520 INFO [M:0;jenkins-hbase4:41275] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-11-02 03:56:10,520 DEBUG [M:0;jenkins-hbase4:41275] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-11-02 03:56:10,520 INFO [M:0;jenkins-hbase4:41275] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-02 03:56:10,520 DEBUG [M:0;jenkins-hbase4:41275] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-02 03:56:10,520 DEBUG [M:0;jenkins-hbase4:41275] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-11-02 03:56:10,520 DEBUG [M:0;jenkins-hbase4:41275] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-02 03:56:10,520 INFO [M:0;jenkins-hbase4:41275] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.92 KB heapSize=599.05 KB 2023-11-02 03:56:10,569 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-11-02 03:56:10,615 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-02 03:56:10,615 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): regionserver:36627-0x101a4e939270001, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-02 03:56:10,615 INFO [RS:0;jenkins-hbase4:36627] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,36627,1698897201574; zookeeper connection closed. 2023-11-02 03:56:10,615 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@4e211329] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@4e211329 2023-11-02 03:56:10,615 INFO [Listener at localhost/43647] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-11-02 03:56:10,933 INFO [M:0;jenkins-hbase4:41275] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.92 KB at sequenceid=1011 (bloomFilter=true), to=hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/ff7e6527d88e4fa99c28b90725068dc4 2023-11-02 03:56:10,937 INFO [M:0;jenkins-hbase4:41275] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ff7e6527d88e4fa99c28b90725068dc4 2023-11-02 03:56:10,938 DEBUG [M:0;jenkins-hbase4:41275] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/ff7e6527d88e4fa99c28b90725068dc4 as hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/ff7e6527d88e4fa99c28b90725068dc4 2023-11-02 03:56:10,942 INFO [M:0;jenkins-hbase4:41275] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ff7e6527d88e4fa99c28b90725068dc4 2023-11-02 03:56:10,942 INFO [M:0;jenkins-hbase4:41275] regionserver.HStore(1080): Added hdfs://localhost:39871/user/jenkins/test-data/4c315cd6-976b-1dad-1cd1-c4e65f1cb7eb/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/ff7e6527d88e4fa99c28b90725068dc4, entries=123, sequenceid=1011, filesize=33.3 K 2023-11-02 03:56:10,943 INFO [M:0;jenkins-hbase4:41275] regionserver.HRegion(2948): Finished flush of dataSize ~512.92 KB/525228, heapSize ~599.03 KB/613408, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 423ms, sequenceid=1011, compaction requested=false 2023-11-02 03:56:10,943 INFO [M:0;jenkins-hbase4:41275] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-11-02 03:56:10,943 DEBUG [M:0;jenkins-hbase4:41275] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-11-02 03:56:10,946 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-11-02 03:56:10,946 INFO [M:0;jenkins-hbase4:41275] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-11-02 03:56:10,947 INFO [M:0;jenkins-hbase4:41275] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41275 2023-11-02 03:56:10,950 DEBUG [M:0;jenkins-hbase4:41275] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,41275,1698897200405 already deleted, retry=false 2023-11-02 03:56:11,051 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-02 03:56:11,051 INFO [M:0;jenkins-hbase4:41275] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,41275,1698897200405; zookeeper connection closed. 2023-11-02 03:56:11,051 DEBUG [Listener at localhost/43647-EventThread] zookeeper.ZKWatcher(600): master:41275-0x101a4e939270000, quorum=127.0.0.1:65185, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-11-02 03:56:11,053 WARN [Listener at localhost/43647] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-11-02 03:56:11,056 INFO [Listener at localhost/43647] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-11-02 03:56:11,161 WARN [BP-301327040-172.31.14.131-1698897197016 heartbeating to localhost/127.0.0.1:39871] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-11-02 03:56:11,161 WARN [BP-301327040-172.31.14.131-1698897197016 heartbeating to localhost/127.0.0.1:39871] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-301327040-172.31.14.131-1698897197016 (Datanode Uuid 3448414c-2f66-47a5-b942-f11f797b9875) service to localhost/127.0.0.1:39871 2023-11-02 03:56:11,163 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/cluster_a48e5c36-4405-e712-b078-c0f3440ea40f/dfs/data/data1/current/BP-301327040-172.31.14.131-1698897197016] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-11-02 03:56:11,164 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/da538169-bbff-9088-c674-75f5718fd438/cluster_a48e5c36-4405-e712-b078-c0f3440ea40f/dfs/data/data2/current/BP-301327040-172.31.14.131-1698897197016] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-11-02 03:56:11,191 INFO [Listener at localhost/43647] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-11-02 03:56:11,323 INFO [Listener at localhost/43647] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-11-02 03:56:11,401 INFO [Listener at localhost/43647] hbase.HBaseTestingUtility(1293): Minicluster is down