2023-02-08 08:53:27,740 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-02-08 08:53:28,004 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca 2023-02-08 08:53:28,019 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-08 08:53:28,019 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/cluster_d5539413-62cb-b490-c3d3-1723c9563de1, deleteOnExit=true 2023-02-08 08:53:28,020 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-08 08:53:28,020 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/test.cache.data in system properties and HBase conf 2023-02-08 08:53:28,021 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/hadoop.tmp.dir in system properties and HBase conf 2023-02-08 08:53:28,021 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/hadoop.log.dir in system properties and HBase conf 2023-02-08 08:53:28,022 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-08 08:53:28,022 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-08 08:53:28,022 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-08 08:53:28,135 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-08 08:53:28,538 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-08 08:53:28,542 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-08 08:53:28,543 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-08 08:53:28,543 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-08 08:53:28,544 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-08 08:53:28,544 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-08 08:53:28,545 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-08 08:53:28,545 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-08 08:53:28,545 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-08 08:53:28,546 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-08 08:53:28,546 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/nfs.dump.dir in system properties and HBase conf 2023-02-08 08:53:28,547 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/java.io.tmpdir in system properties and HBase conf 2023-02-08 08:53:28,547 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-08 08:53:28,547 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-08 08:53:28,547 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-08 08:53:29,060 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-08 08:53:29,064 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-08 08:53:29,364 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-08 08:53:29,545 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-08 08:53:29,566 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-08 08:53:29,609 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-08 08:53:29,644 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/java.io.tmpdir/Jetty_localhost_38135_hdfs____jm9t2w/webapp 2023-02-08 08:53:29,793 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38135 2023-02-08 08:53:29,831 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-08 08:53:29,832 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-08 08:53:30,317 WARN [Listener at localhost/33187] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-08 08:53:30,438 WARN [Listener at localhost/33187] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-08 08:53:30,456 WARN [Listener at localhost/33187] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-08 08:53:30,464 INFO [Listener at localhost/33187] log.Slf4jLog(67): jetty-6.1.26 2023-02-08 08:53:30,469 INFO [Listener at localhost/33187] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/java.io.tmpdir/Jetty_localhost_35625_datanode____.dvoe2q/webapp 2023-02-08 08:53:30,573 INFO [Listener at localhost/33187] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35625 2023-02-08 08:53:30,881 WARN [Listener at localhost/40763] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-08 08:53:31,283 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xca081c7137edfbdf: Processing first storage report for DS-f3681318-d6b4-48d5-b8d4-993d47321769 from datanode 0fbfb9d4-86e7-4262-8b59-c196f8c06b5d 2023-02-08 08:53:31,285 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xca081c7137edfbdf: from storage DS-f3681318-d6b4-48d5-b8d4-993d47321769 node DatanodeRegistration(127.0.0.1:40801, datanodeUuid=0fbfb9d4-86e7-4262-8b59-c196f8c06b5d, infoPort=38151, infoSecurePort=0, ipcPort=40763, storageInfo=lv=-57;cid=testClusterID;nsid=1374576509;c=1675846409146), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-02-08 08:53:31,285 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xca081c7137edfbdf: Processing first storage report for DS-9fc49328-cc7a-4adf-9b37-4465a12f8aff from datanode 0fbfb9d4-86e7-4262-8b59-c196f8c06b5d 2023-02-08 08:53:31,285 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xca081c7137edfbdf: from storage DS-9fc49328-cc7a-4adf-9b37-4465a12f8aff node DatanodeRegistration(127.0.0.1:40801, datanodeUuid=0fbfb9d4-86e7-4262-8b59-c196f8c06b5d, infoPort=38151, infoSecurePort=0, ipcPort=40763, storageInfo=lv=-57;cid=testClusterID;nsid=1374576509;c=1675846409146), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-08 08:53:31,361 DEBUG [Listener at localhost/40763] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca 2023-02-08 08:53:31,429 INFO [Listener at localhost/40763] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/cluster_d5539413-62cb-b490-c3d3-1723c9563de1/zookeeper_0, clientPort=61594, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/cluster_d5539413-62cb-b490-c3d3-1723c9563de1/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/cluster_d5539413-62cb-b490-c3d3-1723c9563de1/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-08 08:53:31,442 INFO [Listener at localhost/40763] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=61594 2023-02-08 08:53:31,449 INFO [Listener at localhost/40763] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 08:53:31,451 INFO [Listener at localhost/40763] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 08:53:32,094 INFO [Listener at localhost/40763] util.FSUtils(479): Created version file at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004 with version=8 2023-02-08 08:53:32,095 INFO [Listener at localhost/40763] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/hbase-staging 2023-02-08 08:53:32,387 INFO [Listener at localhost/40763] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-08 08:53:32,833 INFO [Listener at localhost/40763] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-02-08 08:53:32,864 INFO [Listener at localhost/40763] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 08:53:32,864 INFO [Listener at localhost/40763] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-08 08:53:32,864 INFO [Listener at localhost/40763] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-08 08:53:32,864 INFO [Listener at localhost/40763] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 08:53:32,865 INFO [Listener at localhost/40763] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-08 08:53:33,002 INFO [Listener at localhost/40763] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-08 08:53:33,074 DEBUG [Listener at localhost/40763] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-08 08:53:33,164 INFO [Listener at localhost/40763] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37869 2023-02-08 08:53:33,174 INFO [Listener at localhost/40763] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 08:53:33,176 INFO [Listener at localhost/40763] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 08:53:33,196 INFO [Listener at localhost/40763] zookeeper.RecoverableZooKeeper(93): Process identifier=master:37869 connecting to ZooKeeper ensemble=127.0.0.1:61594 2023-02-08 08:53:33,234 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:378690x0, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-08 08:53:33,237 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:37869-0x1003322ce5d0000 connected 2023-02-08 08:53:33,262 DEBUG [Listener at localhost/40763] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-08 08:53:33,263 DEBUG [Listener at localhost/40763] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 08:53:33,266 DEBUG [Listener at localhost/40763] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-08 08:53:33,274 DEBUG [Listener at localhost/40763] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37869 2023-02-08 08:53:33,274 DEBUG [Listener at localhost/40763] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37869 2023-02-08 08:53:33,275 DEBUG [Listener at localhost/40763] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37869 2023-02-08 08:53:33,275 DEBUG [Listener at localhost/40763] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37869 2023-02-08 08:53:33,275 DEBUG [Listener at localhost/40763] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37869 2023-02-08 08:53:33,281 INFO [Listener at localhost/40763] master.HMaster(439): hbase.rootdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004, hbase.cluster.distributed=false 2023-02-08 08:53:33,349 INFO [Listener at localhost/40763] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-02-08 08:53:33,349 INFO [Listener at localhost/40763] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 08:53:33,349 INFO [Listener at localhost/40763] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-08 08:53:33,349 INFO [Listener at localhost/40763] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-08 08:53:33,349 INFO [Listener at localhost/40763] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-08 08:53:33,350 INFO [Listener at localhost/40763] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-08 08:53:33,354 INFO [Listener at localhost/40763] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-08 08:53:33,357 INFO [Listener at localhost/40763] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42239 2023-02-08 08:53:33,359 INFO [Listener at localhost/40763] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-08 08:53:33,365 DEBUG [Listener at localhost/40763] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-08 08:53:33,366 INFO [Listener at localhost/40763] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 08:53:33,369 INFO [Listener at localhost/40763] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 08:53:33,370 INFO [Listener at localhost/40763] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42239 connecting to ZooKeeper ensemble=127.0.0.1:61594 2023-02-08 08:53:33,374 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:422390x0, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-08 08:53:33,375 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:42239-0x1003322ce5d0001 connected 2023-02-08 08:53:33,375 DEBUG [Listener at localhost/40763] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-08 08:53:33,376 DEBUG [Listener at localhost/40763] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 08:53:33,377 DEBUG [Listener at localhost/40763] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-08 08:53:33,378 DEBUG [Listener at localhost/40763] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42239 2023-02-08 08:53:33,378 DEBUG [Listener at localhost/40763] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42239 2023-02-08 08:53:33,379 DEBUG [Listener at localhost/40763] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42239 2023-02-08 08:53:33,379 DEBUG [Listener at localhost/40763] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42239 2023-02-08 08:53:33,380 DEBUG [Listener at localhost/40763] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42239 2023-02-08 08:53:33,382 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,37869,1675846412232 2023-02-08 08:53:33,390 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-08 08:53:33,392 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,37869,1675846412232 2023-02-08 08:53:33,409 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-08 08:53:33,409 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-08 08:53:33,409 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 08:53:33,410 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-08 08:53:33,411 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,37869,1675846412232 from backup master directory 2023-02-08 08:53:33,411 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-08 08:53:33,416 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,37869,1675846412232 2023-02-08 08:53:33,416 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-08 08:53:33,416 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-08 08:53:33,417 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,37869,1675846412232 2023-02-08 08:53:33,419 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-08 08:53:33,420 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-08 08:53:33,500 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/hbase.id with ID: f37b0169-25f5-43cf-a6e6-1ab895001374 2023-02-08 08:53:33,540 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 08:53:33,553 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 08:53:33,589 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x29bc2cbd to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:53:33,620 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@22ef1b8b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:53:33,642 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-08 08:53:33,644 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-08 08:53:33,660 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-08 08:53:33,660 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-08 08:53:33,662 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-08 08:53:33,665 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-08 08:53:33,667 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-08 08:53:33,698 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/MasterData/data/master/store-tmp 2023-02-08 08:53:33,722 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:53:33,723 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-08 08:53:33,723 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 08:53:33,723 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 08:53:33,723 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-08 08:53:33,723 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 08:53:33,723 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 08:53:33,724 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675846413723Disabling compacts and flushes for region at 1675846413723Disabling writes for close at 1675846413723Writing region close event to WAL at 1675846413723Closed at 1675846413723 2023-02-08 08:53:33,725 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/MasterData/WALs/jenkins-hbase4.apache.org,37869,1675846412232 2023-02-08 08:53:33,745 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37869%2C1675846412232, suffix=, logDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/MasterData/WALs/jenkins-hbase4.apache.org,37869,1675846412232, archiveDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/MasterData/oldWALs, maxLogs=10 2023-02-08 08:53:33,808 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40801,DS-f3681318-d6b4-48d5-b8d4-993d47321769,DISK] 2023-02-08 08:53:33,815 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-08 08:53:33,878 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/MasterData/WALs/jenkins-hbase4.apache.org,37869,1675846412232/jenkins-hbase4.apache.org%2C37869%2C1675846412232.1675846413755 2023-02-08 08:53:33,879 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40801,DS-f3681318-d6b4-48d5-b8d4-993d47321769,DISK]] 2023-02-08 08:53:33,880 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:53:33,880 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:53:33,884 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-08 08:53:33,886 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-08 08:53:33,951 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-08 08:53:33,957 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-08 08:53:33,981 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-08 08:53:33,994 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:53:34,000 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-08 08:53:34,002 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-08 08:53:34,016 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-08 08:53:34,020 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:53:34,021 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64405784, jitterRate=-0.04027903079986572}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 08:53:34,022 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1675846413915Initializing all the Stores at 1675846413918 (+3 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846413919 (+1 ms)Cleaning up temporary data from old regions at 1675846414008 (+89 ms)Cleaning up detritus from prior splits at 1675846414009 (+1 ms)Region opened successfully at 1675846414021 (+12 ms) 2023-02-08 08:53:34,023 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-08 08:53:34,042 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-08 08:53:34,042 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-08 08:53:34,044 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-08 08:53:34,046 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-08 08:53:34,078 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 31 msec 2023-02-08 08:53:34,078 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-08 08:53:34,103 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-08 08:53:34,108 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-08 08:53:34,131 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-08 08:53:34,135 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-08 08:53:34,137 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-08 08:53:34,141 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-08 08:53:34,145 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-08 08:53:34,148 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 08:53:34,149 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-08 08:53:34,150 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-08 08:53:34,161 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-08 08:53:34,171 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-08 08:53:34,171 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-08 08:53:34,171 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 08:53:34,171 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,37869,1675846412232, sessionid=0x1003322ce5d0000, setting cluster-up flag (Was=false) 2023-02-08 08:53:34,186 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 08:53:34,192 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-08 08:53:34,193 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37869,1675846412232 2023-02-08 08:53:34,197 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 08:53:34,204 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-08 08:53:34,205 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37869,1675846412232 2023-02-08 08:53:34,208 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.hbase-snapshot/.tmp 2023-02-08 08:53:34,284 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(952): ClusterId : f37b0169-25f5-43cf-a6e6-1ab895001374 2023-02-08 08:53:34,289 DEBUG [RS:0;jenkins-hbase4:42239] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-08 08:53:34,296 DEBUG [RS:0;jenkins-hbase4:42239] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-08 08:53:34,296 DEBUG [RS:0;jenkins-hbase4:42239] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-08 08:53:34,300 DEBUG [RS:0;jenkins-hbase4:42239] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-08 08:53:34,301 DEBUG [RS:0;jenkins-hbase4:42239] zookeeper.ReadOnlyZKClient(139): Connect 0x43be4382 to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:53:34,305 DEBUG [RS:0;jenkins-hbase4:42239] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4d79a585, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:53:34,306 DEBUG [RS:0;jenkins-hbase4:42239] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@494628f2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-08 08:53:34,320 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-08 08:53:34,334 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-08 08:53:34,334 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-08 08:53:34,334 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-08 08:53:34,334 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-08 08:53:34,334 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-08 08:53:34,334 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:53:34,334 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-08 08:53:34,335 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:53:34,336 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1675846444336 2023-02-08 08:53:34,338 DEBUG [RS:0;jenkins-hbase4:42239] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:42239 2023-02-08 08:53:34,338 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-08 08:53:34,340 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-08 08:53:34,341 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-08 08:53:34,343 INFO [RS:0;jenkins-hbase4:42239] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-08 08:53:34,343 INFO [RS:0;jenkins-hbase4:42239] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-08 08:53:34,343 DEBUG [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(1023): About to register with Master. 2023-02-08 08:53:34,345 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-08 08:53:34,346 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,37869,1675846412232 with isa=jenkins-hbase4.apache.org/172.31.14.131:42239, startcode=1675846413348 2023-02-08 08:53:34,351 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-08 08:53:34,360 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-08 08:53:34,361 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-08 08:53:34,361 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-08 08:53:34,361 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-08 08:53:34,362 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-08 08:53:34,363 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-08 08:53:34,365 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-08 08:53:34,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-08 08:53:34,369 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-08 08:53:34,370 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-08 08:53:34,372 DEBUG [RS:0;jenkins-hbase4:42239] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-08 08:53:34,373 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675846414373,5,FailOnTimeoutGroup] 2023-02-08 08:53:34,377 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675846414377,5,FailOnTimeoutGroup] 2023-02-08 08:53:34,377 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-08 08:53:34,377 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-08 08:53:34,378 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-08 08:53:34,379 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-08 08:53:34,380 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-08 08:53:34,381 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-08 08:53:34,382 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004 2023-02-08 08:53:34,393 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:53:34,396 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-08 08:53:34,399 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/info 2023-02-08 08:53:34,400 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-08 08:53:34,402 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:53:34,402 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-08 08:53:34,405 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/rep_barrier 2023-02-08 08:53:34,406 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-08 08:53:34,407 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:53:34,407 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-08 08:53:34,410 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/table 2023-02-08 08:53:34,410 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-08 08:53:34,411 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:53:34,413 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740 2023-02-08 08:53:34,414 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740 2023-02-08 08:53:34,417 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-08 08:53:34,419 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-08 08:53:34,423 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:53:34,424 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59794898, jitterRate=-0.10898658633232117}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-08 08:53:34,424 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1675846414394Initializing all the Stores at 1675846414396 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675846414396Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846414396Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675846414396Cleaning up temporary data from old regions at 1675846414415 (+19 ms)Cleaning up detritus from prior splits at 1675846414416 (+1 ms)Region opened successfully at 1675846414424 (+8 ms) 2023-02-08 08:53:34,425 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-08 08:53:34,425 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-08 08:53:34,425 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-08 08:53:34,425 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-08 08:53:34,425 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-08 08:53:34,426 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-08 08:53:34,426 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675846414425Disabling compacts and flushes for region at 1675846414425Disabling writes for close at 1675846414425Writing region close event to WAL at 1675846414426 (+1 ms)Closed at 1675846414426 2023-02-08 08:53:34,431 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-08 08:53:34,431 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-08 08:53:34,432 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59393, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-08 08:53:34,440 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-08 08:53:34,445 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:34,455 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-08 08:53:34,457 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-08 08:53:34,462 DEBUG [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004 2023-02-08 08:53:34,462 DEBUG [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:33187 2023-02-08 08:53:34,463 DEBUG [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-08 08:53:34,467 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 08:53:34,468 DEBUG [RS:0;jenkins-hbase4:42239] zookeeper.ZKUtil(162): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:34,468 WARN [RS:0;jenkins-hbase4:42239] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-08 08:53:34,468 INFO [RS:0;jenkins-hbase4:42239] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-08 08:53:34,469 DEBUG [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(1947): logDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/WALs/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:34,470 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42239,1675846413348] 2023-02-08 08:53:34,477 DEBUG [RS:0;jenkins-hbase4:42239] zookeeper.ZKUtil(162): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:34,486 DEBUG [RS:0;jenkins-hbase4:42239] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-08 08:53:34,493 INFO [RS:0;jenkins-hbase4:42239] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-08 08:53:34,512 INFO [RS:0;jenkins-hbase4:42239] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-08 08:53:34,515 INFO [RS:0;jenkins-hbase4:42239] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:53:34,515 INFO [RS:0;jenkins-hbase4:42239] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:53:34,516 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-08 08:53:34,523 INFO [RS:0;jenkins-hbase4:42239] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-02-08 08:53:34,523 DEBUG [RS:0;jenkins-hbase4:42239] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:53:34,523 DEBUG [RS:0;jenkins-hbase4:42239] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:53:34,523 DEBUG [RS:0;jenkins-hbase4:42239] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:53:34,523 DEBUG [RS:0;jenkins-hbase4:42239] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:53:34,524 DEBUG [RS:0;jenkins-hbase4:42239] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:53:34,524 DEBUG [RS:0;jenkins-hbase4:42239] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-08 08:53:34,524 DEBUG [RS:0;jenkins-hbase4:42239] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:53:34,524 DEBUG [RS:0;jenkins-hbase4:42239] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:53:34,524 DEBUG [RS:0;jenkins-hbase4:42239] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:53:34,524 DEBUG [RS:0;jenkins-hbase4:42239] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-08 08:53:34,525 INFO [RS:0;jenkins-hbase4:42239] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 08:53:34,525 INFO [RS:0;jenkins-hbase4:42239] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-08 08:53:34,525 INFO [RS:0;jenkins-hbase4:42239] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-08 08:53:34,540 INFO [RS:0;jenkins-hbase4:42239] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-08 08:53:34,542 INFO [RS:0;jenkins-hbase4:42239] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42239,1675846413348-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:53:34,557 INFO [RS:0;jenkins-hbase4:42239] regionserver.Replication(203): jenkins-hbase4.apache.org,42239,1675846413348 started 2023-02-08 08:53:34,557 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,42239,1675846413348, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42239, sessionid=0x1003322ce5d0001 2023-02-08 08:53:34,557 DEBUG [RS:0;jenkins-hbase4:42239] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-08 08:53:34,557 DEBUG [RS:0;jenkins-hbase4:42239] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:34,557 DEBUG [RS:0;jenkins-hbase4:42239] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42239,1675846413348' 2023-02-08 08:53:34,557 DEBUG [RS:0;jenkins-hbase4:42239] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:34,558 DEBUG [RS:0;jenkins-hbase4:42239] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:34,558 DEBUG [RS:0;jenkins-hbase4:42239] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-08 08:53:34,558 DEBUG [RS:0;jenkins-hbase4:42239] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-08 08:53:34,558 DEBUG [RS:0;jenkins-hbase4:42239] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:34,558 DEBUG [RS:0;jenkins-hbase4:42239] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42239,1675846413348' 2023-02-08 08:53:34,558 DEBUG [RS:0;jenkins-hbase4:42239] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-08 08:53:34,559 DEBUG [RS:0;jenkins-hbase4:42239] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-08 08:53:34,559 DEBUG [RS:0;jenkins-hbase4:42239] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-08 08:53:34,559 INFO [RS:0;jenkins-hbase4:42239] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-08 08:53:34,559 INFO [RS:0;jenkins-hbase4:42239] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-08 08:53:34,609 DEBUG [jenkins-hbase4:37869] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-02-08 08:53:34,611 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42239,1675846413348, state=OPENING 2023-02-08 08:53:34,618 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-08 08:53:34,622 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 08:53:34,623 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-08 08:53:34,626 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:53:34,670 INFO [RS:0;jenkins-hbase4:42239] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42239%2C1675846413348, suffix=, logDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/WALs/jenkins-hbase4.apache.org,42239,1675846413348, archiveDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/oldWALs, maxLogs=32 2023-02-08 08:53:34,687 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40801,DS-f3681318-d6b4-48d5-b8d4-993d47321769,DISK] 2023-02-08 08:53:34,691 INFO [RS:0;jenkins-hbase4:42239] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/WALs/jenkins-hbase4.apache.org,42239,1675846413348/jenkins-hbase4.apache.org%2C42239%2C1675846413348.1675846414672 2023-02-08 08:53:34,691 DEBUG [RS:0;jenkins-hbase4:42239] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40801,DS-f3681318-d6b4-48d5-b8d4-993d47321769,DISK]] 2023-02-08 08:53:34,807 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:34,810 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-08 08:53:34,813 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:32852, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-08 08:53:34,825 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-08 08:53:34,825 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-08 08:53:34,829 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42239%2C1675846413348.meta, suffix=.meta, logDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/WALs/jenkins-hbase4.apache.org,42239,1675846413348, archiveDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/oldWALs, maxLogs=32 2023-02-08 08:53:34,845 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40801,DS-f3681318-d6b4-48d5-b8d4-993d47321769,DISK] 2023-02-08 08:53:34,849 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/WALs/jenkins-hbase4.apache.org,42239,1675846413348/jenkins-hbase4.apache.org%2C42239%2C1675846413348.meta.1675846414830.meta 2023-02-08 08:53:34,849 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40801,DS-f3681318-d6b4-48d5-b8d4-993d47321769,DISK]] 2023-02-08 08:53:34,850 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:53:34,851 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-08 08:53:34,868 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-08 08:53:34,872 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-08 08:53:34,878 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-08 08:53:34,878 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:53:34,878 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-08 08:53:34,878 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-08 08:53:34,881 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-08 08:53:34,883 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/info 2023-02-08 08:53:34,883 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/info 2023-02-08 08:53:34,883 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-08 08:53:34,884 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:53:34,885 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-08 08:53:34,886 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/rep_barrier 2023-02-08 08:53:34,886 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/rep_barrier 2023-02-08 08:53:34,887 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-08 08:53:34,888 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:53:34,888 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-08 08:53:34,890 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/table 2023-02-08 08:53:34,890 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/table 2023-02-08 08:53:34,890 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-08 08:53:34,891 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:53:34,893 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740 2023-02-08 08:53:34,895 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740 2023-02-08 08:53:34,899 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-08 08:53:34,902 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-08 08:53:34,903 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64414774, jitterRate=-0.04014506936073303}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-08 08:53:34,903 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1675846414878Writing region info on filesystem at 1675846414879 (+1 ms)Initializing all the Stores at 1675846414880 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675846414881 (+1 ms)Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1675846414881Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675846414881Cleaning up temporary data from old regions at 1675846414897 (+16 ms)Cleaning up detritus from prior splits at 1675846414898 (+1 ms)Running coprocessor post-open hooks at 1675846414903 (+5 ms)Region opened successfully at 1675846414903 2023-02-08 08:53:34,913 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1675846414800 2023-02-08 08:53:34,929 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-08 08:53:34,929 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-08 08:53:34,930 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,42239,1675846413348, state=OPEN 2023-02-08 08:53:34,932 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-08 08:53:34,932 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-08 08:53:34,936 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-08 08:53:34,936 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,42239,1675846413348 in 306 msec 2023-02-08 08:53:34,943 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-08 08:53:34,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 498 msec 2023-02-08 08:53:34,949 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 697 msec 2023-02-08 08:53:34,949 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1675846413406, completionTime=-1 2023-02-08 08:53:34,949 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-02-08 08:53:34,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-08 08:53:35,007 DEBUG [hconnection-0x2fd4c29f-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:53:35,011 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:32862, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:53:35,027 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=1 2023-02-08 08:53:35,027 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1675846475027 2023-02-08 08:53:35,027 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1675846535027 2023-02-08 08:53:35,027 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 78 msec 2023-02-08 08:53:35,049 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37869,1675846412232-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:53:35,050 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37869,1675846412232-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-08 08:53:35,050 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37869,1675846412232-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-08 08:53:35,051 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:37869, period=300000, unit=MILLISECONDS is enabled. 2023-02-08 08:53:35,052 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-08 08:53:35,057 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-08 08:53:35,064 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-08 08:53:35,066 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-08 08:53:35,075 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-08 08:53:35,077 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:53:35,080 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:53:35,100 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/hbase/namespace/d6a9c249ca6bd89fbc448e7f6a1c9903 2023-02-08 08:53:35,102 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/hbase/namespace/d6a9c249ca6bd89fbc448e7f6a1c9903 empty. 2023-02-08 08:53:35,103 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/hbase/namespace/d6a9c249ca6bd89fbc448e7f6a1c9903 2023-02-08 08:53:35,103 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-08 08:53:35,523 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-08 08:53:35,526 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => d6a9c249ca6bd89fbc448e7f6a1c9903, NAME => 'hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp 2023-02-08 08:53:35,938 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:53:35,938 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing d6a9c249ca6bd89fbc448e7f6a1c9903, disabling compactions & flushes 2023-02-08 08:53:35,938 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903. 2023-02-08 08:53:35,938 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903. 2023-02-08 08:53:35,938 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903. after waiting 0 ms 2023-02-08 08:53:35,939 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903. 2023-02-08 08:53:35,939 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903. 2023-02-08 08:53:35,939 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for d6a9c249ca6bd89fbc448e7f6a1c9903: Waiting for close lock at 1675846415938Disabling compacts and flushes for region at 1675846415938Disabling writes for close at 1675846415939 (+1 ms)Writing region close event to WAL at 1675846415939Closed at 1675846415939 2023-02-08 08:53:35,943 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:53:35,958 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675846415946"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846415946"}]},"ts":"1675846415946"} 2023-02-08 08:53:35,987 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 08:53:35,989 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:53:35,994 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846415989"}]},"ts":"1675846415989"} 2023-02-08 08:53:35,999 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-08 08:53:36,008 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=d6a9c249ca6bd89fbc448e7f6a1c9903, ASSIGN}] 2023-02-08 08:53:36,011 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=d6a9c249ca6bd89fbc448e7f6a1c9903, ASSIGN 2023-02-08 08:53:36,012 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=d6a9c249ca6bd89fbc448e7f6a1c9903, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=false 2023-02-08 08:53:36,163 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=d6a9c249ca6bd89fbc448e7f6a1c9903, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:36,164 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675846416163"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846416163"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846416163"}]},"ts":"1675846416163"} 2023-02-08 08:53:36,169 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure d6a9c249ca6bd89fbc448e7f6a1c9903, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:53:36,330 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903. 2023-02-08 08:53:36,330 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => d6a9c249ca6bd89fbc448e7f6a1c9903, NAME => 'hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:53:36,331 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace d6a9c249ca6bd89fbc448e7f6a1c9903 2023-02-08 08:53:36,332 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:53:36,332 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for d6a9c249ca6bd89fbc448e7f6a1c9903 2023-02-08 08:53:36,332 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for d6a9c249ca6bd89fbc448e7f6a1c9903 2023-02-08 08:53:36,334 INFO [StoreOpener-d6a9c249ca6bd89fbc448e7f6a1c9903-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region d6a9c249ca6bd89fbc448e7f6a1c9903 2023-02-08 08:53:36,336 DEBUG [StoreOpener-d6a9c249ca6bd89fbc448e7f6a1c9903-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/namespace/d6a9c249ca6bd89fbc448e7f6a1c9903/info 2023-02-08 08:53:36,336 DEBUG [StoreOpener-d6a9c249ca6bd89fbc448e7f6a1c9903-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/namespace/d6a9c249ca6bd89fbc448e7f6a1c9903/info 2023-02-08 08:53:36,337 INFO [StoreOpener-d6a9c249ca6bd89fbc448e7f6a1c9903-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d6a9c249ca6bd89fbc448e7f6a1c9903 columnFamilyName info 2023-02-08 08:53:36,338 INFO [StoreOpener-d6a9c249ca6bd89fbc448e7f6a1c9903-1] regionserver.HStore(310): Store=d6a9c249ca6bd89fbc448e7f6a1c9903/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:53:36,339 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/namespace/d6a9c249ca6bd89fbc448e7f6a1c9903 2023-02-08 08:53:36,340 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/namespace/d6a9c249ca6bd89fbc448e7f6a1c9903 2023-02-08 08:53:36,344 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for d6a9c249ca6bd89fbc448e7f6a1c9903 2023-02-08 08:53:36,348 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/namespace/d6a9c249ca6bd89fbc448e7f6a1c9903/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:53:36,348 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened d6a9c249ca6bd89fbc448e7f6a1c9903; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63742019, jitterRate=-0.05016990005970001}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 08:53:36,349 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for d6a9c249ca6bd89fbc448e7f6a1c9903: Running coprocessor pre-open hook at 1675846416332Writing region info on filesystem at 1675846416332Initializing all the Stores at 1675846416334 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1675846416334Cleaning up temporary data from old regions at 1675846416340 (+6 ms)Cleaning up detritus from prior splits at 1675846416341 (+1 ms)Running coprocessor post-open hooks at 1675846416348 (+7 ms)Region opened successfully at 1675846416348 2023-02-08 08:53:36,350 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903., pid=6, masterSystemTime=1675846416323 2023-02-08 08:53:36,354 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903. 2023-02-08 08:53:36,354 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903. 2023-02-08 08:53:36,355 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=d6a9c249ca6bd89fbc448e7f6a1c9903, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:36,356 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1675846416355"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846416355"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846416355"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846416355"}]},"ts":"1675846416355"} 2023-02-08 08:53:36,362 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-08 08:53:36,362 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure d6a9c249ca6bd89fbc448e7f6a1c9903, server=jenkins-hbase4.apache.org,42239,1675846413348 in 190 msec 2023-02-08 08:53:36,365 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-08 08:53:36,365 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=d6a9c249ca6bd89fbc448e7f6a1c9903, ASSIGN in 354 msec 2023-02-08 08:53:36,366 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:53:36,367 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846416366"}]},"ts":"1675846416366"} 2023-02-08 08:53:36,369 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-08 08:53:36,372 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:53:36,378 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 1.3060 sec 2023-02-08 08:53:36,378 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-08 08:53:36,381 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-08 08:53:36,381 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 08:53:36,415 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-08 08:53:36,429 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-08 08:53:36,434 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 27 msec 2023-02-08 08:53:36,438 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-08 08:53:36,448 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-08 08:53:36,455 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 16 msec 2023-02-08 08:53:36,464 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-08 08:53:36,467 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-08 08:53:36,467 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 3.050sec 2023-02-08 08:53:36,469 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-08 08:53:36,470 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-08 08:53:36,470 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-08 08:53:36,471 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37869,1675846412232-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-08 08:53:36,473 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37869,1675846412232-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-08 08:53:36,486 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-08 08:53:36,489 DEBUG [Listener at localhost/40763] zookeeper.ReadOnlyZKClient(139): Connect 0x3c1e7f16 to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:53:36,493 DEBUG [Listener at localhost/40763] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@672b468, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:53:36,507 DEBUG [hconnection-0x86f282a-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:53:36,518 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:32874, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:53:36,528 INFO [Listener at localhost/40763] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,37869,1675846412232 2023-02-08 08:53:36,529 DEBUG [Listener at localhost/40763] zookeeper.ReadOnlyZKClient(139): Connect 0x64b944ad to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:53:36,533 DEBUG [Listener at localhost/40763] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@33fc1e18, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:53:36,534 INFO [Listener at localhost/40763] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 08:53:36,555 INFO [Listener at localhost/40763] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=209, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=269, ProcessCount=168, AvailableMemoryMB=5190 2023-02-08 08:53:36,559 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:36,559 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:36,566 DEBUG [Listener at localhost/40763] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 08:53:36,570 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55086, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 08:53:36,583 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-08 08:53:36,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:36,587 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:53:36,589 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:53:36,591 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-02-08 08:53:36,592 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6 2023-02-08 08:53:36,593 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6 empty. 2023-02-08 08:53:36,595 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6 2023-02-08 08:53:36,595 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-02-08 08:53:36,599 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:36,599 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:36,601 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-08 08:53:36,612 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-02-08 08:53:36,614 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7671): creating {ENCODED => a6defd68663718a57a591a97ac1ac7c6, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp 2023-02-08 08:53:36,705 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:36,705 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:36,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-08 08:53:36,908 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:36,908 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:36,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-08 08:53:37,026 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:53:37,026 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1603): Closing a6defd68663718a57a591a97ac1ac7c6, disabling compactions & flushes 2023-02-08 08:53:37,026 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:37,027 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:37,027 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. after waiting 0 ms 2023-02-08 08:53:37,027 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:37,027 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:37,027 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1557): Region close journal for a6defd68663718a57a591a97ac1ac7c6: Waiting for close lock at 1675846417026Disabling compacts and flushes for region at 1675846417026Disabling writes for close at 1675846417027 (+1 ms)Writing region close event to WAL at 1675846417027Closed at 1675846417027 2023-02-08 08:53:37,031 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:53:37,033 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675846417033"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846417033"}]},"ts":"1675846417033"} 2023-02-08 08:53:37,036 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 08:53:37,037 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:53:37,037 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846417037"}]},"ts":"1675846417037"} 2023-02-08 08:53:37,039 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-02-08 08:53:37,044 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=a6defd68663718a57a591a97ac1ac7c6, ASSIGN}] 2023-02-08 08:53:37,046 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=a6defd68663718a57a591a97ac1ac7c6, ASSIGN 2023-02-08 08:53:37,048 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=a6defd68663718a57a591a97ac1ac7c6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=false 2023-02-08 08:53:37,199 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=a6defd68663718a57a591a97ac1ac7c6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:37,199 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675846417199"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846417199"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846417199"}]},"ts":"1675846417199"} 2023-02-08 08:53:37,203 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure a6defd68663718a57a591a97ac1ac7c6, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:53:37,210 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:37,210 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:37,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-08 08:53:37,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:37,362 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a6defd68663718a57a591a97ac1ac7c6, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:53:37,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly a6defd68663718a57a591a97ac1ac7c6 2023-02-08 08:53:37,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:53:37,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a6defd68663718a57a591a97ac1ac7c6 2023-02-08 08:53:37,363 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a6defd68663718a57a591a97ac1ac7c6 2023-02-08 08:53:37,366 INFO [StoreOpener-a6defd68663718a57a591a97ac1ac7c6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a6defd68663718a57a591a97ac1ac7c6 2023-02-08 08:53:37,368 DEBUG [StoreOpener-a6defd68663718a57a591a97ac1ac7c6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1 2023-02-08 08:53:37,368 DEBUG [StoreOpener-a6defd68663718a57a591a97ac1ac7c6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1 2023-02-08 08:53:37,371 INFO [StoreOpener-a6defd68663718a57a591a97ac1ac7c6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a6defd68663718a57a591a97ac1ac7c6 columnFamilyName family1 2023-02-08 08:53:37,374 INFO [StoreOpener-a6defd68663718a57a591a97ac1ac7c6-1] regionserver.HStore(310): Store=a6defd68663718a57a591a97ac1ac7c6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:53:37,376 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6 2023-02-08 08:53:37,377 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6 2023-02-08 08:53:37,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a6defd68663718a57a591a97ac1ac7c6 2023-02-08 08:53:37,385 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:53:37,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a6defd68663718a57a591a97ac1ac7c6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67987191, jitterRate=0.01308809220790863}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 08:53:37,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a6defd68663718a57a591a97ac1ac7c6: Running coprocessor pre-open hook at 1675846417363Writing region info on filesystem at 1675846417363Initializing all the Stores at 1675846417364 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675846417364Cleaning up temporary data from old regions at 1675846417377 (+13 ms)Cleaning up detritus from prior splits at 1675846417378 (+1 ms)Running coprocessor post-open hooks at 1675846417386 (+8 ms)Region opened successfully at 1675846417386 2023-02-08 08:53:37,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6., pid=11, masterSystemTime=1675846417356 2023-02-08 08:53:37,390 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:37,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:37,391 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=a6defd68663718a57a591a97ac1ac7c6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:37,392 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675846417391"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846417391"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846417391"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846417391"}]},"ts":"1675846417391"} 2023-02-08 08:53:37,397 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-08 08:53:37,398 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure a6defd68663718a57a591a97ac1ac7c6, server=jenkins-hbase4.apache.org,42239,1675846413348 in 192 msec 2023-02-08 08:53:37,401 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-08 08:53:37,402 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=a6defd68663718a57a591a97ac1ac7c6, ASSIGN in 354 msec 2023-02-08 08:53:37,403 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:53:37,403 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846417403"}]},"ts":"1675846417403"} 2023-02-08 08:53:37,405 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-02-08 08:53:37,408 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:53:37,410 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 825 msec 2023-02-08 08:53:37,712 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:37,712 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:37,714 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-08 08:53:37,714 INFO [Listener at localhost/40763] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-02-08 08:53:37,732 DEBUG [hconnection-0x130f47b7-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:53:37,735 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:32888, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:53:37,873 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:37,873 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:37,876 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:53:37,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:37,884 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 08:53:37,884 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:37,885 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-08 08:53:37,885 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-08 08:53:37,885 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:37,885 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:37,887 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:37,887 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:37,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:37,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:37,887 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:37,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:37,887 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:53:37,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:37,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:53:37,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:37,889 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:37,890 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:37,891 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-08 08:53:37,891 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:37,891 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-08 08:53:37,892 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:53:37,892 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:53:37,892 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:37,893 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. started... 2023-02-08 08:53:37,894 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6defd68663718a57a591a97ac1ac7c6 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-08 08:53:38,470 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511302913802fa563486cb0065e64e5893e3c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511302913802fa563486cb0065e64e5893e3c 2023-02-08 08:53:38,885 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/dfda8c77b57046bf8728aa22307210b7 2023-02-08 08:53:38,896 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/dfda8c77b57046bf8728aa22307210b7 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/dfda8c77b57046bf8728aa22307210b7 2023-02-08 08:53:38,906 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/dfda8c77b57046bf8728aa22307210b7, entries=1000, sequenceid=5, filesize=175.8 K 2023-02-08 08:53:38,908 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for a6defd68663718a57a591a97ac1ac7c6 in 1015ms, sequenceid=5, compaction requested=false 2023-02-08 08:53:38,910 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 08:53:38,914 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6defd68663718a57a591a97ac1ac7c6: 2023-02-08 08:53:38,914 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:38,914 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:53:38,915 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:53:38,915 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:38,915 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-08 08:53:38,915 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-08 08:53:38,918 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:38,918 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:38,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:38,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:38,919 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,919 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:53:38,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:38,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:38,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:38,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:38,921 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-08 08:53:38,921 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-08 08:53:38,921 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d47d841[Count = 0] remaining members to acquire global barrier 2023-02-08 08:53:38,921 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,924 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,925 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-08 08:53:38,925 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-08 08:53:38,925 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:53:38,925 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:38,925 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:53:38,927 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-08 08:53:38,927 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:38,927 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:38,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:38,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:38,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:38,927 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-08 08:53:38,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:38,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:38,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:38,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:38,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:38,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:53:38,932 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-08 08:53:38,932 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-08 08:53:38,932 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:53:38,932 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:53:38,932 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,932 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:53:38,934 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,934 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,934 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:38,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:38,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:38,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:38,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:38,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:38,936 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:38,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:38,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,938 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:38,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:38,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:38,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:38,949 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:38,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:38,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:38,949 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:38,950 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:38,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:38,950 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:38,950 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,950 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:38,950 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,950 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,950 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:38,950 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:38,951 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 08:53:38,951 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:53:38,951 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-08 08:53:38,953 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-08 08:53:38,953 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:53:39,053 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:53:39,054 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:39,054 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:39,058 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:53:39,069 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:39,069 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:39,070 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:53:39,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-08 08:53:39,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,074 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 08:53:39,074 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:39,074 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-08 08:53:39,074 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-08 08:53:39,075 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,075 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,077 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:39,077 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:39,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:39,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:39,078 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:39,078 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:53:39,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,078 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:53:39,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,079 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,079 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-08 08:53:39,079 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,079 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-08 08:53:39,079 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:39,080 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-08 08:53:39,080 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:53:39,080 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:53:39,080 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:39,080 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. started... 2023-02-08 08:53:39,080 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6defd68663718a57a591a97ac1ac7c6 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-08 08:53:39,117 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128c1a0774ff4a74c59a41b2b8778cf4c5b to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128c1a0774ff4a74c59a41b2b8778cf4c5b 2023-02-08 08:53:39,528 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/33092bd6e30c40ffad796fe7bac59251 2023-02-08 08:53:39,537 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/33092bd6e30c40ffad796fe7bac59251 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/33092bd6e30c40ffad796fe7bac59251 2023-02-08 08:53:39,546 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/33092bd6e30c40ffad796fe7bac59251, entries=1, sequenceid=9, filesize=5.0 K 2023-02-08 08:53:39,548 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for a6defd68663718a57a591a97ac1ac7c6 in 468ms, sequenceid=9, compaction requested=false 2023-02-08 08:53:39,548 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6defd68663718a57a591a97ac1ac7c6: 2023-02-08 08:53:39,548 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:39,548 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:53:39,549 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:53:39,549 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:39,549 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-08 08:53:39,549 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-08 08:53:39,551 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:39,551 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:39,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:39,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:39,552 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,552 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:53:39,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:39,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:39,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:39,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:39,554 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-08 08:53:39,554 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7bade226[Count = 0] remaining members to acquire global barrier 2023-02-08 08:53:39,554 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-08 08:53:39,554 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,555 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,555 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,555 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-08 08:53:39,555 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-08 08:53:39,556 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:53:39,556 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:39,556 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:53:39,557 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-08 08:53:39,557 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:39,557 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:39,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:39,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:39,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:39,558 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-08 08:53:39,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:39,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:39,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:39,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:39,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:39,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:53:39,561 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-08 08:53:39,561 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-08 08:53:39,561 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:53:39,561 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:53:39,561 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,562 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:53:39,568 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,568 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,568 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:39,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:39,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:39,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,568 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:39,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:39,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:39,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:39,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:39,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,570 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:39,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:39,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:39,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:39,574 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:39,574 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:39,574 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:39,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:39,574 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 08:53:39,575 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:53:39,574 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,574 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:39,574 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:39,575 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:39,575 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:39,575 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:39,575 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,575 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-08 08:53:39,575 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-08 08:53:39,575 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:53:39,676 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:53:39,676 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:39,676 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:39,677 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:53:39,685 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:39,685 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:39,686 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:53:39,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-08 08:53:39,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,689 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 08:53:39,689 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:39,689 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-08 08:53:39,689 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-08 08:53:39,690 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,690 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,691 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:39,691 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:39,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:39,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:39,692 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:39,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,692 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:53:39,693 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:53:39,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,693 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,693 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-08 08:53:39,693 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:39,694 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-08 08:53:39,694 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:39,694 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-08 08:53:39,694 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:53:39,694 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:53:39,694 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:39,694 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. started... 2023-02-08 08:53:39,694 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6defd68663718a57a591a97ac1ac7c6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:53:40,122 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120146e27e6af47841319e6d5fc1e25bc44e to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120146e27e6af47841319e6d5fc1e25bc44e 2023-02-08 08:53:40,387 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 08:53:40,490 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-08 08:53:40,491 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-08 08:53:40,536 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/5f3ce8f9849c4ff38f177ec9897d1823 2023-02-08 08:53:40,547 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/5f3ce8f9849c4ff38f177ec9897d1823 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/5f3ce8f9849c4ff38f177ec9897d1823 2023-02-08 08:53:40,556 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/5f3ce8f9849c4ff38f177ec9897d1823, entries=1, sequenceid=13, filesize=5.0 K 2023-02-08 08:53:40,557 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a6defd68663718a57a591a97ac1ac7c6 in 863ms, sequenceid=13, compaction requested=true 2023-02-08 08:53:40,557 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6defd68663718a57a591a97ac1ac7c6: 2023-02-08 08:53:40,557 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:40,557 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:53:40,558 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:53:40,558 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:40,558 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-08 08:53:40,558 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-08 08:53:40,560 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,560 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:40,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:40,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:40,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:40,560 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,561 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:53:40,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:40,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:40,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:40,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:40,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-08 08:53:40,563 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5f097b67[Count = 0] remaining members to acquire global barrier 2023-02-08 08:53:40,563 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-08 08:53:40,563 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,565 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,565 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-08 08:53:40,565 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-08 08:53:40,565 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:53:40,565 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:40,566 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:53:40,568 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-08 08:53:40,568 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:40,568 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:40,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:40,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:40,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:40,568 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-08 08:53:40,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:40,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:40,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:40,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:40,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:53:40,573 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-08 08:53:40,573 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-08 08:53:40,573 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:53:40,573 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:53:40,573 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,573 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:53:40,575 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,575 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:40,575 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:40,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:40,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:40,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:40,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:40,577 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:40,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:40,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,581 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:40,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:40,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:40,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:40,586 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:40,586 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,588 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,588 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:40,588 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:40,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:40,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:40,588 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:40,588 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-08 08:53:40,588 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:40,588 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 08:53:40,589 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:53:40,589 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,589 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,589 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:40,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:40,589 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-08 08:53:40,590 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:53:40,690 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:53:40,690 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:40,690 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:40,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:53:40,700 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:40,700 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:40,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:53:40,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-08 08:53:40,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,705 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 08:53:40,705 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:40,705 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-08 08:53:40,705 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-08 08:53:40,706 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,706 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,708 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:40,708 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:40,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:40,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:40,709 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:40,709 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:53:40,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,709 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:53:40,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,710 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,710 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-08 08:53:40,710 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:40,711 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-08 08:53:40,711 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:40,711 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-08 08:53:40,711 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:53:40,711 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:53:40,711 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:40,711 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. started... 2023-02-08 08:53:40,711 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6defd68663718a57a591a97ac1ac7c6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:53:41,143 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120573b22f9e6cd34dc2a8d93473767799e7 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120573b22f9e6cd34dc2a8d93473767799e7 2023-02-08 08:53:41,559 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/bb5c59da4acd46e89bd3369a40f23959 2023-02-08 08:53:41,568 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/bb5c59da4acd46e89bd3369a40f23959 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/bb5c59da4acd46e89bd3369a40f23959 2023-02-08 08:53:41,578 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/bb5c59da4acd46e89bd3369a40f23959, entries=1, sequenceid=17, filesize=5.0 K 2023-02-08 08:53:41,580 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a6defd68663718a57a591a97ac1ac7c6 in 869ms, sequenceid=17, compaction requested=true 2023-02-08 08:53:41,580 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6defd68663718a57a591a97ac1ac7c6: 2023-02-08 08:53:41,580 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:41,580 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:53:41,581 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:53:41,581 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:41,581 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-08 08:53:41,581 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-08 08:53:41,583 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:41,583 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:41,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:41,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:41,584 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,584 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:53:41,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:41,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:41,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:41,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:41,586 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-08 08:53:41,586 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@534c4d92[Count = 0] remaining members to acquire global barrier 2023-02-08 08:53:41,586 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-08 08:53:41,586 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,587 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,587 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-08 08:53:41,587 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-08 08:53:41,587 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:53:41,588 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:41,588 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:53:41,589 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-08 08:53:41,589 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:41,589 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:41,589 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-08 08:53:41,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:41,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:41,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:41,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:41,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:41,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:41,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:53:41,594 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-08 08:53:41,594 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-08 08:53:41,594 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:53:41,594 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:53:41,594 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,594 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:53:41,597 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,597 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:41,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,597 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:41,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:41,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:41,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:41,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:41,598 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:41,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:41,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:41,601 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:41,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:41,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:41,606 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:41,606 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,606 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,606 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:41,606 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:41,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:41,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:41,606 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:41,606 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:41,606 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-08 08:53:41,606 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 08:53:41,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:53:41,607 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:41,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:41,607 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,607 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,608 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-08 08:53:41,608 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:53:41,708 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:53:41,708 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:41,708 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:41,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:53:41,717 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:41,718 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:41,719 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:53:41,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-08 08:53:41,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,721 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 08:53:41,721 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:41,722 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-08 08:53:41,722 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-08 08:53:41,723 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,723 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,724 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:41,724 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:41,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:41,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:41,724 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:41,724 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:53:41,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,725 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:53:41,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,726 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,726 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-08 08:53:41,726 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:41,726 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-08 08:53:41,726 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:41,726 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-08 08:53:41,726 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:53:41,727 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:53:41,727 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:41,727 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. started... 2023-02-08 08:53:41,727 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6defd68663718a57a591a97ac1ac7c6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:53:42,149 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228fb156effaffe4b07a85ae679f11ea253 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228fb156effaffe4b07a85ae679f11ea253 2023-02-08 08:53:42,382 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-08 08:53:42,383 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-08 08:53:42,561 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/eda1576258ce4ce9b1283dc50a70667e 2023-02-08 08:53:42,570 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/eda1576258ce4ce9b1283dc50a70667e as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/eda1576258ce4ce9b1283dc50a70667e 2023-02-08 08:53:42,577 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/eda1576258ce4ce9b1283dc50a70667e, entries=1, sequenceid=21, filesize=5.0 K 2023-02-08 08:53:42,578 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a6defd68663718a57a591a97ac1ac7c6 in 851ms, sequenceid=21, compaction requested=true 2023-02-08 08:53:42,578 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6defd68663718a57a591a97ac1ac7c6: 2023-02-08 08:53:42,578 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:42,579 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:53:42,579 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:53:42,579 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:42,579 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-08 08:53:42,579 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-08 08:53:42,770 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:42,770 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:42,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:42,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:42,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:42,771 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:42,771 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:53:42,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:42,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:42,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:42,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:42,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:42,773 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-08 08:53:42,773 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60fa026a[Count = 0] remaining members to acquire global barrier 2023-02-08 08:53:42,773 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-08 08:53:42,773 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:42,808 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:42,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:42,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:42,809 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-08 08:53:42,809 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-08 08:53:42,809 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:53:42,809 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:42,809 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:53:42,862 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-08 08:53:42,862 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:42,863 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:42,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:42,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:42,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:42,863 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-08 08:53:42,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:42,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:42,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:42,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:42,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:42,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:42,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:42,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:53:42,866 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-08 08:53:42,866 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-08 08:53:42,866 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:53:42,866 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:53:42,866 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:42,866 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:53:42,924 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:42,924 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:42,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:42,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:42,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:42,924 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:42,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:42,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:42,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:42,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:42,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:42,925 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:42,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:42,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:42,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:42,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:42,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:42,927 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:42,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:42,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:42,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:43,017 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:43,017 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:43,018 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 08:53:43,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:43,018 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:53:43,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:43,018 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:43,018 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:43,018 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:43,018 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:43,018 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:43,019 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:43,019 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:43,019 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:43,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:43,019 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-08 08:53:43,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:43,019 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:53:43,119 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:53:43,119 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:43,119 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:43,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:53:43,127 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:43,127 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:43,128 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:53:43,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-08 08:53:43,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:43,131 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 08:53:43,131 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:43,131 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-08 08:53:43,131 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-08 08:53:43,132 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:43,132 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:43,216 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:43,216 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:43,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:43,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:43,217 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:43,217 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:53:43,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:43,217 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:43,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:53:43,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:43,218 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:43,218 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-08 08:53:43,218 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:43,218 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-08 08:53:43,218 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:43,219 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-08 08:53:43,219 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:53:43,219 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:53:43,219 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:43,219 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. started... 2023-02-08 08:53:43,220 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6defd68663718a57a591a97ac1ac7c6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:53:43,641 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231191d9009f3f5419d9c2396a2f7cf07a4 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231191d9009f3f5419d9c2396a2f7cf07a4 2023-02-08 08:53:44,050 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/df6cc0aaf61a400c94720a13d06cb3dc 2023-02-08 08:53:44,060 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/df6cc0aaf61a400c94720a13d06cb3dc as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/df6cc0aaf61a400c94720a13d06cb3dc 2023-02-08 08:53:44,067 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/df6cc0aaf61a400c94720a13d06cb3dc, entries=1, sequenceid=25, filesize=5.0 K 2023-02-08 08:53:44,069 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a6defd68663718a57a591a97ac1ac7c6 in 849ms, sequenceid=25, compaction requested=true 2023-02-08 08:53:44,069 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6defd68663718a57a591a97ac1ac7c6: 2023-02-08 08:53:44,069 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:44,070 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:53:44,070 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:53:44,070 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:44,070 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-08 08:53:44,070 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-08 08:53:44,072 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,072 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:44,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:44,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:44,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:44,073 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,073 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:53:44,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:44,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:44,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:44,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:44,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-08 08:53:44,075 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74527081[Count = 0] remaining members to acquire global barrier 2023-02-08 08:53:44,075 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-08 08:53:44,075 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,076 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,077 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-08 08:53:44,077 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-08 08:53:44,077 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:53:44,077 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:44,077 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:53:44,078 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-08 08:53:44,078 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:44,078 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:44,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:44,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:44,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:44,078 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-08 08:53:44,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:44,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:44,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:44,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:44,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:44,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:53:44,082 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-08 08:53:44,082 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-08 08:53:44,082 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:53:44,083 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:53:44,083 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,083 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:53:44,086 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,086 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,086 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:44,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:44,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:44,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,087 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:44,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:44,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:44,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:44,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:44,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,088 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:44,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:44,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:44,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:44,093 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:44,093 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,093 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:44,093 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,093 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 08:53:44,094 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:44,093 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:44,094 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:44,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:44,094 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:53:44,094 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,094 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:44,094 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,094 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-08 08:53:44,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:44,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:44,095 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-08 08:53:44,095 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:53:44,195 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:53:44,195 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:44,195 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:44,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:53:44,203 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:44,203 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:44,204 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:53:44,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-08 08:53:44,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,206 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 08:53:44,206 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:44,207 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-08 08:53:44,207 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-08 08:53:44,207 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,207 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,209 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:44,209 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:44,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:44,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:44,209 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:44,209 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:53:44,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,210 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:53:44,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,211 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,211 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-08 08:53:44,211 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:44,211 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-08 08:53:44,211 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:44,212 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-08 08:53:44,212 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:53:44,212 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:53:44,212 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:44,212 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. started... 2023-02-08 08:53:44,212 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6defd68663718a57a591a97ac1ac7c6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:53:44,634 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010151a3429666d243f7832f2c790c2fd2a6 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010151a3429666d243f7832f2c790c2fd2a6 2023-02-08 08:53:45,043 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/e83e8b8163c64239b3eb5ed9db2ba456 2023-02-08 08:53:45,051 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/e83e8b8163c64239b3eb5ed9db2ba456 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/e83e8b8163c64239b3eb5ed9db2ba456 2023-02-08 08:53:45,059 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/e83e8b8163c64239b3eb5ed9db2ba456, entries=1, sequenceid=29, filesize=5.0 K 2023-02-08 08:53:45,061 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a6defd68663718a57a591a97ac1ac7c6 in 849ms, sequenceid=29, compaction requested=true 2023-02-08 08:53:45,061 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6defd68663718a57a591a97ac1ac7c6: 2023-02-08 08:53:45,061 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:45,061 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:53:45,062 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:53:45,062 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:45,062 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-08 08:53:45,062 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-08 08:53:45,064 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:45,064 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:45,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:45,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:45,064 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,064 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:53:45,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:45,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:45,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:45,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:45,066 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-08 08:53:45,066 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6094ff2f[Count = 0] remaining members to acquire global barrier 2023-02-08 08:53:45,066 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-08 08:53:45,066 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,068 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,068 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,068 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-08 08:53:45,069 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-08 08:53:45,069 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:45,069 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:53:45,069 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:53:45,070 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-08 08:53:45,070 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:45,070 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:45,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:45,070 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-08 08:53:45,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:45,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:45,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:45,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:45,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:45,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:45,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:45,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:53:45,073 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-08 08:53:45,073 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-08 08:53:45,073 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:53:45,073 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:53:45,073 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,073 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:53:45,075 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,075 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:45,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:45,075 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:45,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:45,075 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:45,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:45,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:45,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:45,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,077 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:45,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:45,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:45,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:45,083 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:45,083 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,083 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:45,083 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:45,084 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-08 08:53:45,084 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:45,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 08:53:45,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:53:45,084 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:45,083 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:45,084 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:45,084 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,084 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,085 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-08 08:53:45,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:45,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:45,085 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:53:45,185 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:53:45,185 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:45,185 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:45,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:53:45,193 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:45,193 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:45,194 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:53:45,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-08 08:53:45,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,196 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 08:53:45,196 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:45,197 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-08 08:53:45,197 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-08 08:53:45,197 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,197 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,199 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:45,199 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:45,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:45,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:45,199 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:45,199 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:53:45,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,200 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:53:45,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,200 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,200 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-08 08:53:45,200 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:45,200 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-08 08:53:45,200 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:45,201 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-08 08:53:45,201 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:53:45,201 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:53:45,201 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:45,201 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. started... 2023-02-08 08:53:45,201 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6defd68663718a57a591a97ac1ac7c6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:53:45,622 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601038e83c3fd1be14506aa01822d10bb3cd9 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601038e83c3fd1be14506aa01822d10bb3cd9 2023-02-08 08:53:46,031 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/849a73e5cc144928a5ad5b2ca000ff44 2023-02-08 08:53:46,039 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/849a73e5cc144928a5ad5b2ca000ff44 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/849a73e5cc144928a5ad5b2ca000ff44 2023-02-08 08:53:46,046 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/849a73e5cc144928a5ad5b2ca000ff44, entries=1, sequenceid=33, filesize=5.0 K 2023-02-08 08:53:46,047 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a6defd68663718a57a591a97ac1ac7c6 in 846ms, sequenceid=33, compaction requested=true 2023-02-08 08:53:46,047 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6defd68663718a57a591a97ac1ac7c6: 2023-02-08 08:53:46,047 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:46,047 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:53:46,048 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:53:46,048 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:46,048 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-08 08:53:46,048 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-08 08:53:46,050 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,050 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:46,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:46,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:46,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:46,050 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,050 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:53:46,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:46,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:46,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:46,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:46,052 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-08 08:53:46,052 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@51051e72[Count = 0] remaining members to acquire global barrier 2023-02-08 08:53:46,052 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-08 08:53:46,052 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,053 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,053 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-08 08:53:46,053 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-08 08:53:46,053 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:53:46,054 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:46,054 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:53:46,056 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-08 08:53:46,056 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:46,056 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:46,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:46,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:46,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:46,056 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-08 08:53:46,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:46,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:46,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:46,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:46,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:46,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:53:46,059 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-08 08:53:46,059 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-08 08:53:46,059 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:53:46,059 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:53:46,059 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,059 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:53:46,061 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,061 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:46,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,061 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:46,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:46,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:46,062 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:46,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:46,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,063 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:46,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:46,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:46,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:46,065 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:46,065 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,065 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,065 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:46,065 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,065 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:46,065 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-08 08:53:46,065 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 08:53:46,066 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:53:46,065 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:46,065 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:46,066 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,066 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:46,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:46,066 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-08 08:53:46,066 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:53:46,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:46,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:46,167 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:53:46,167 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:46,167 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:46,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:53:46,175 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:46,175 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:46,176 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:53:46,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-08 08:53:46,179 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,179 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 08:53:46,179 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:46,180 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-08 08:53:46,180 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-08 08:53:46,180 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,180 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,182 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:46,182 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:46,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:46,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:46,182 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:46,182 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:53:46,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,183 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:53:46,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,183 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,187 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-08 08:53:46,187 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:46,187 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-08 08:53:46,187 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:46,187 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-08 08:53:46,188 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:53:46,188 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:53:46,188 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:46,188 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. started... 2023-02-08 08:53:46,188 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6defd68663718a57a591a97ac1ac7c6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:53:46,610 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509076f6436f0afc14f06ad4ac2dc9c398f5f to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201509076f6436f0afc14f06ad4ac2dc9c398f5f 2023-02-08 08:53:47,019 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/b5d39941fa7e42d19c5581427e09bd0a 2023-02-08 08:53:47,027 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/b5d39941fa7e42d19c5581427e09bd0a as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/b5d39941fa7e42d19c5581427e09bd0a 2023-02-08 08:53:47,034 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/b5d39941fa7e42d19c5581427e09bd0a, entries=1, sequenceid=37, filesize=5.0 K 2023-02-08 08:53:47,036 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a6defd68663718a57a591a97ac1ac7c6 in 848ms, sequenceid=37, compaction requested=true 2023-02-08 08:53:47,036 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6defd68663718a57a591a97ac1ac7c6: 2023-02-08 08:53:47,036 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:47,036 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:53:47,036 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:53:47,036 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:47,036 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-08 08:53:47,037 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-08 08:53:47,039 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,039 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:47,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:47,039 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,039 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:53:47,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:47,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:47,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:47,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:47,041 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-08 08:53:47,041 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@61dcaf6a[Count = 0] remaining members to acquire global barrier 2023-02-08 08:53:47,041 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-08 08:53:47,041 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,042 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,042 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-08 08:53:47,043 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-08 08:53:47,043 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:53:47,043 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:47,043 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:53:47,044 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:47,044 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-08 08:53:47,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:47,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:47,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:47,044 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:47,044 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-08 08:53:47,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:47,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:47,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:47,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:47,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:47,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:53:47,047 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-08 08:53:47,047 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-08 08:53:47,047 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:53:47,047 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:53:47,047 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,047 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:53:47,049 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,049 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:47,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:47,049 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:47,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:47,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:47,049 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:47,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:47,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:47,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,051 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:47,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:47,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:47,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:47,054 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:47,054 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,054 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:47,054 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:47,055 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:47,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:47,055 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-08 08:53:47,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 08:53:47,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:53:47,055 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:47,055 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:47,055 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:47,055 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:47,055 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,056 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-08 08:53:47,056 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:53:47,156 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:53:47,156 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:47,156 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:47,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:53:47,164 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:47,164 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:47,165 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:53:47,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-08 08:53:47,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,167 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 08:53:47,167 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:47,167 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-08 08:53:47,167 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-08 08:53:47,168 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,168 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,169 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:47,169 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:47,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:47,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:47,170 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:47,170 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:53:47,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,170 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:53:47,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,171 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,171 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-08 08:53:47,171 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:47,171 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-08 08:53:47,171 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:47,172 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-08 08:53:47,173 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:53:47,173 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:53:47,173 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:47,173 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. started... 2023-02-08 08:53:47,173 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a6defd68663718a57a591a97ac1ac7c6 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:53:47,593 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511202c95dd3a10ca410fb995ee31fe91b131 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511202c95dd3a10ca410fb995ee31fe91b131 2023-02-08 08:53:48,001 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/2f24d557c6fb43e4917dac3ae697a502 2023-02-08 08:53:48,009 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/2f24d557c6fb43e4917dac3ae697a502 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/2f24d557c6fb43e4917dac3ae697a502 2023-02-08 08:53:48,015 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/2f24d557c6fb43e4917dac3ae697a502, entries=1, sequenceid=41, filesize=5.0 K 2023-02-08 08:53:48,016 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for a6defd68663718a57a591a97ac1ac7c6 in 843ms, sequenceid=41, compaction requested=true 2023-02-08 08:53:48,016 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a6defd68663718a57a591a97ac1ac7c6: 2023-02-08 08:53:48,016 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:48,017 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:53:48,017 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:53:48,017 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:48,017 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-08 08:53:48,017 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-08 08:53:48,019 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:48,019 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:48,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:48,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:48,019 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:48,019 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:53:48,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:48,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:48,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:48,020 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-08 08:53:48,021 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66d0da8[Count = 0] remaining members to acquire global barrier 2023-02-08 08:53:48,021 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-08 08:53:48,021 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,022 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,022 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,022 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-08 08:53:48,022 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:48,022 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:53:48,022 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-08 08:53:48,022 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:53:48,028 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-08 08:53:48,028 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:48,028 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:48,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:48,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:48,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:48,028 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-08 08:53:48,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:48,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:48,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:48,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:48,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:48,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:53:48,032 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-08 08:53:48,032 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-08 08:53:48,032 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:53:48,032 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:53:48,032 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,032 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:53:48,034 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,034 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:48,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:48,034 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:48,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:48,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:48,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:48,034 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:48,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:48,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,036 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:48,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:48,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:48,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:48,038 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:48,038 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:48,038 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:48,038 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:48,038 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,038 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:48,038 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:48,038 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-08 08:53:48,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:48,038 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-08 08:53:48,039 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:53:48,038 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,039 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,039 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:48,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:48,039 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-08 08:53:48,039 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:53:48,139 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:53:48,139 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:48,139 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:48,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:53:48,145 DEBUG [Listener at localhost/40763] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-08 08:53:48,148 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48098, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-08 08:53:48,149 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37869] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:48,161 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-08 08:53:48,162 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-08 08:53:48,163 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:53:48,163 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675847028163 2023-02-08 08:53:48,166 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 08:53:48,171 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-02-08 08:53:48,171 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-08 08:53:48,172 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] zookeeper.ReadOnlyZKClient(139): Connect 0x77b35877 to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:53:48,176 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2a4707d0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:53:48,177 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 08:53:49,081 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010363ab859e96dc4bc3afcb3099222e82f2 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010363ab859e96dc4bc3afcb3099222e82f2 2023-02-08 08:53:49,082 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-08 08:53:49,101 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:53:49,103 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43382, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:53:49,112 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:49,112 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:49,113 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 08:53:49,115 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48106, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 08:53:49,148 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/53b55cc0edc74e819d8a2bdc7b7d02f7 first=Optional[r3] last=Optional[r6] 2023-02-08 08:53:49,161 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/53b55cc0edc74e819d8a2bdc7b7d02f7}] 2023-02-08 08:53:49,175 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/53b55cc0edc74e819d8a2bdc7b7d02f7 for inclusion in a6defd68663718a57a591a97ac1ac7c6/family1 2023-02-08 08:53:49,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-08 08:53:49,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 08:53:49,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for a6defd68663718a57a591a97ac1ac7c6: 2023-02-08 08:53:49,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/53b55cc0edc74e819d8a2bdc7b7d02f7 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__q7bm38n5q7903cfmqklj66n4p45j4fvoqnf06qq9n0vcomijtnbc070srticqp4c/family1/53b55cc0edc74e819d8a2bdc7b7d02f7 2023-02-08 08:53:49,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__q7bm38n5q7903cfmqklj66n4p45j4fvoqnf06qq9n0vcomijtnbc070srticqp4c/family1/53b55cc0edc74e819d8a2bdc7b7d02f7 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/26bdb6f86a91441fa5bb1da2e09c85bf_SeqId_44_ 2023-02-08 08:53:49,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__q7bm38n5q7903cfmqklj66n4p45j4fvoqnf06qq9n0vcomijtnbc070srticqp4c/family1/53b55cc0edc74e819d8a2bdc7b7d02f7 into a6defd68663718a57a591a97ac1ac7c6/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/26bdb6f86a91441fa5bb1da2e09c85bf_SeqId_44_ - updating store file list. 2023-02-08 08:53:49,193 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/26bdb6f86a91441fa5bb1da2e09c85bf_SeqId_44_ into a6defd68663718a57a591a97ac1ac7c6/family1 2023-02-08 08:53:49,193 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__q7bm38n5q7903cfmqklj66n4p45j4fvoqnf06qq9n0vcomijtnbc070srticqp4c/family1/53b55cc0edc74e819d8a2bdc7b7d02f7 into a6defd68663718a57a591a97ac1ac7c6/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/26bdb6f86a91441fa5bb1da2e09c85bf_SeqId_44_) 2023-02-08 08:53:49,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__q7bm38n5q7903cfmqklj66n4p45j4fvoqnf06qq9n0vcomijtnbc070srticqp4c/family1/53b55cc0edc74e819d8a2bdc7b7d02f7 2023-02-08 08:53:49,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-08 08:53:49,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-08 08:53:49,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(1912): a6defd68663718a57a591a97ac1ac7c6/family1 is initiating minor compaction 2023-02-08 08:53:49,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1339b682; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:53:49,206 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a6defd68663718a57a591a97ac1ac7c6/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:49,206 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/33092bd6e30c40ffad796fe7bac59251, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/5f3ce8f9849c4ff38f177ec9897d1823, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/bb5c59da4acd46e89bd3369a40f23959, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/eda1576258ce4ce9b1283dc50a70667e, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/df6cc0aaf61a400c94720a13d06cb3dc, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/e83e8b8163c64239b3eb5ed9db2ba456, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/849a73e5cc144928a5ad5b2ca000ff44, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/b5d39941fa7e42d19c5581427e09bd0a, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/2f24d557c6fb43e4917dac3ae697a502, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/26bdb6f86a91441fa5bb1da2e09c85bf_SeqId_44_] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp, totalSize=50.2 K 2023-02-08 08:53:49,207 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 33092bd6e30c40ffad796fe7bac59251, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-08 08:53:49,207 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 5f3ce8f9849c4ff38f177ec9897d1823, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-08 08:53:49,207 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting bb5c59da4acd46e89bd3369a40f23959, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-08 08:53:49,207 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting eda1576258ce4ce9b1283dc50a70667e, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-08 08:53:49,207 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting df6cc0aaf61a400c94720a13d06cb3dc, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-08 08:53:49,207 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting e83e8b8163c64239b3eb5ed9db2ba456, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-08 08:53:49,207 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 849a73e5cc144928a5ad5b2ca000ff44, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-08 08:53:49,207 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting b5d39941fa7e42d19c5581427e09bd0a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-08 08:53:49,207 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 2f24d557c6fb43e4917dac3ae697a502, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-08 08:53:49,207 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 26bdb6f86a91441fa5bb1da2e09c85bf_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-08 08:53:49,220 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-08 08:53:49,222 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:53:49,227 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228fb156effaffe4b07a85ae679f11ea253 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228fb156effaffe4b07a85ae679f11ea253 2023-02-08 08:53:49,229 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231191d9009f3f5419d9c2396a2f7cf07a4 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151231191d9009f3f5419d9c2396a2f7cf07a4 2023-02-08 08:53:49,231 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010151a3429666d243f7832f2c790c2fd2a6 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2016010151a3429666d243f7832f2c790c2fd2a6 2023-02-08 08:53:49,233 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601038e83c3fd1be14506aa01822d10bb3cd9 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601038e83c3fd1be14506aa01822d10bb3cd9 2023-02-08 08:53:49,233 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-02-08 08:53:49,233 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-08 08:53:49,234 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 08:53:49,234 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 08:53:49,234 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 08:53:49,234 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 08:53:49,234 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 08:53:49,234 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 08:53:49,234 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 08:53:49,234 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 08:53:49,254 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] throttle.PressureAwareThroughputController(145): a6defd68663718a57a591a97ac1ac7c6#family1#compaction#10 average throughput is 0.36 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:53:50,085 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/f7b94e46e4164c68b19dbfc6c473ad1c as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/f7b94e46e4164c68b19dbfc6c473ad1c 2023-02-08 08:53:50,100 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in a6defd68663718a57a591a97ac1ac7c6/family1 of a6defd68663718a57a591a97ac1ac7c6 into f7b94e46e4164c68b19dbfc6c473ad1c(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:53:50,100 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a6defd68663718a57a591a97ac1ac7c6: 2023-02-08 08:53:50,101 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1675846429204; duration=0sec 2023-02-08 08:53:50,103 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:53:50,122 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205d0e8daf172c64e2db5bfb7c770b36f6b to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205d0e8daf172c64e2db5bfb7c770b36f6b 2023-02-08 08:53:50,124 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-08 08:53:50,137 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/33092bd6e30c40ffad796fe7bac59251, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/5f3ce8f9849c4ff38f177ec9897d1823, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/bb5c59da4acd46e89bd3369a40f23959, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/eda1576258ce4ce9b1283dc50a70667e, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/df6cc0aaf61a400c94720a13d06cb3dc, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/e83e8b8163c64239b3eb5ed9db2ba456, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/849a73e5cc144928a5ad5b2ca000ff44, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/b5d39941fa7e42d19c5581427e09bd0a, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/2f24d557c6fb43e4917dac3ae697a502, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/26bdb6f86a91441fa5bb1da2e09c85bf_SeqId_44_] to archive 2023-02-08 08:53:50,139 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:53:50,142 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/33092bd6e30c40ffad796fe7bac59251 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/33092bd6e30c40ffad796fe7bac59251 2023-02-08 08:53:50,143 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/5f3ce8f9849c4ff38f177ec9897d1823 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/5f3ce8f9849c4ff38f177ec9897d1823 2023-02-08 08:53:50,145 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/bb5c59da4acd46e89bd3369a40f23959 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/bb5c59da4acd46e89bd3369a40f23959 2023-02-08 08:53:50,146 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/eda1576258ce4ce9b1283dc50a70667e to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/eda1576258ce4ce9b1283dc50a70667e 2023-02-08 08:53:50,148 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/df6cc0aaf61a400c94720a13d06cb3dc to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/df6cc0aaf61a400c94720a13d06cb3dc 2023-02-08 08:53:50,150 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/e83e8b8163c64239b3eb5ed9db2ba456 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/e83e8b8163c64239b3eb5ed9db2ba456 2023-02-08 08:53:50,152 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/849a73e5cc144928a5ad5b2ca000ff44 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/849a73e5cc144928a5ad5b2ca000ff44 2023-02-08 08:53:50,154 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/b5d39941fa7e42d19c5581427e09bd0a to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/b5d39941fa7e42d19c5581427e09bd0a 2023-02-08 08:53:50,155 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/2f24d557c6fb43e4917dac3ae697a502 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/2f24d557c6fb43e4917dac3ae697a502 2023-02-08 08:53:50,157 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/26bdb6f86a91441fa5bb1da2e09c85bf_SeqId_44_ to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/26bdb6f86a91441fa5bb1da2e09c85bf_SeqId_44_ 2023-02-08 08:53:50,166 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:50,168 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:50,180 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/2cdc7a7fa2e746a89598b868e97c4496 first=Optional[r00] last=Optional[r2] 2023-02-08 08:53:50,181 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/2cdc7a7fa2e746a89598b868e97c4496}] 2023-02-08 08:53:50,184 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/2cdc7a7fa2e746a89598b868e97c4496 for inclusion in a6defd68663718a57a591a97ac1ac7c6/family1 2023-02-08 08:53:50,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-08 08:53:50,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 08:53:50,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for a6defd68663718a57a591a97ac1ac7c6: 2023-02-08 08:53:50,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/2cdc7a7fa2e746a89598b868e97c4496 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__2umvlo4a5eao5qhj7s2bp9vi3d26q7ms6on0lpoqf3r8pv8rnoor98n222a8huqu/family1/2cdc7a7fa2e746a89598b868e97c4496 2023-02-08 08:53:50,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__2umvlo4a5eao5qhj7s2bp9vi3d26q7ms6on0lpoqf3r8pv8rnoor98n222a8huqu/family1/2cdc7a7fa2e746a89598b868e97c4496 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/e3462b60047c490b9d3f2720f3bc4c01_SeqId_47_ 2023-02-08 08:53:50,193 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__2umvlo4a5eao5qhj7s2bp9vi3d26q7ms6on0lpoqf3r8pv8rnoor98n222a8huqu/family1/2cdc7a7fa2e746a89598b868e97c4496 into a6defd68663718a57a591a97ac1ac7c6/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/e3462b60047c490b9d3f2720f3bc4c01_SeqId_47_ - updating store file list. 2023-02-08 08:53:50,201 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/e3462b60047c490b9d3f2720f3bc4c01_SeqId_47_ into a6defd68663718a57a591a97ac1ac7c6/family1 2023-02-08 08:53:50,201 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__2umvlo4a5eao5qhj7s2bp9vi3d26q7ms6on0lpoqf3r8pv8rnoor98n222a8huqu/family1/2cdc7a7fa2e746a89598b868e97c4496 into a6defd68663718a57a591a97ac1ac7c6/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/e3462b60047c490b9d3f2720f3bc4c01_SeqId_47_) 2023-02-08 08:53:50,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__2umvlo4a5eao5qhj7s2bp9vi3d26q7ms6on0lpoqf3r8pv8rnoor98n222a8huqu/family1/2cdc7a7fa2e746a89598b868e97c4496 2023-02-08 08:53:50,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-08 08:53:50,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-08 08:53:50,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(1912): a6defd68663718a57a591a97ac1ac7c6/family1 is initiating minor compaction (all files) 2023-02-08 08:53:50,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@21c78a4d; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:53:50,204 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of a6defd68663718a57a591a97ac1ac7c6/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:50,204 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/dfda8c77b57046bf8728aa22307210b7, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/f7b94e46e4164c68b19dbfc6c473ad1c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/e3462b60047c490b9d3f2720f3bc4c01_SeqId_47_] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp, totalSize=358.6 K 2023-02-08 08:53:50,204 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting dfda8c77b57046bf8728aa22307210b7, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878836535 2023-02-08 08:53:50,205 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting f7b94e46e4164c68b19dbfc6c473ad1c, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621236535 2023-02-08 08:53:50,205 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting e3462b60047c490b9d3f2720f3bc4c01_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878836535 2023-02-08 08:53:50,206 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-08 08:53:50,208 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:53:50,210 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511302913802fa563486cb0065e64e5893e3c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511302913802fa563486cb0065e64e5893e3c 2023-02-08 08:53:50,213 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120146e27e6af47841319e6d5fc1e25bc44e to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120146e27e6af47841319e6d5fc1e25bc44e 2023-02-08 08:53:50,214 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120573b22f9e6cd34dc2a8d93473767799e7 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015120573b22f9e6cd34dc2a8d93473767799e7 2023-02-08 08:53:50,215 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-02-08 08:53:50,215 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-08 08:53:50,215 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 08:53:50,215 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-08 08:53:50,216 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:53:50,217 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-08 08:53:50,219 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 2.0630 sec 2023-02-08 08:53:50,252 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] throttle.PressureAwareThroughputController(145): a6defd68663718a57a591a97ac1ac7c6#family1#compaction#11 average throughput is 5.33 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:53:50,378 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:50,378 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:50,385 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-08 08:53:50,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:50,812 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-02-08 08:53:50,813 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-02-08 08:53:50,813 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-02-08 08:53:50,818 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-02-08 08:53:50,830 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=a6defd68663718a57a591a97ac1ac7c6, REOPEN/MOVE}] 2023-02-08 08:53:50,831 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=a6defd68663718a57a591a97ac1ac7c6, REOPEN/MOVE 2023-02-08 08:53:50,833 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=a6defd68663718a57a591a97ac1ac7c6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:50,833 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675846430832"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846430832"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846430832"}]},"ts":"1675846430832"} 2023-02-08 08:53:50,835 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure a6defd68663718a57a591a97ac1ac7c6, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:53:50,993 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close a6defd68663718a57a591a97ac1ac7c6 2023-02-08 08:53:50,994 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a6defd68663718a57a591a97ac1ac7c6, disabling compactions & flushes 2023-02-08 08:53:50,994 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:51,488 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/.tmp/family1/5deba825d9c64eb2b1703ad565fc6768 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/5deba825d9c64eb2b1703ad565fc6768 2023-02-08 08:53:51,496 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in a6defd68663718a57a591a97ac1ac7c6/family1 of a6defd68663718a57a591a97ac1ac7c6 into 5deba825d9c64eb2b1703ad565fc6768(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 08:53:51,497 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for a6defd68663718a57a591a97ac1ac7c6: 2023-02-08 08:53:51,497 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:51,497 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:51,497 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. after waiting 0 ms 2023-02-08 08:53:51,497 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:51,497 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1675846430203; duration=1sec 2023-02-08 08:53:51,497 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:53:51,499 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/dfda8c77b57046bf8728aa22307210b7, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/f7b94e46e4164c68b19dbfc6c473ad1c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/e3462b60047c490b9d3f2720f3bc4c01_SeqId_47_] to archive 2023-02-08 08:53:51,500 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:53:51,502 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/dfda8c77b57046bf8728aa22307210b7 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/dfda8c77b57046bf8728aa22307210b7 2023-02-08 08:53:51,504 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/f7b94e46e4164c68b19dbfc6c473ad1c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/f7b94e46e4164c68b19dbfc6c473ad1c 2023-02-08 08:53:51,505 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/e3462b60047c490b9d3f2720f3bc4c01_SeqId_47_ to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/e3462b60047c490b9d3f2720f3bc4c01_SeqId_47_ 2023-02-08 08:53:51,511 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-08 08:53:51,512 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:51,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a6defd68663718a57a591a97ac1ac7c6: Waiting for close lock at 1675846430994Running coprocessor pre-close hooks at 1675846430994Disabling compacts and flushes for region at 1675846430994Disabling writes for close at 1675846431497 (+503 ms)Writing region close event to WAL at 1675846431508 (+11 ms)Running coprocessor post-close hooks at 1675846431512 (+4 ms)Closed at 1675846431512 2023-02-08 08:53:51,512 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: a6defd68663718a57a591a97ac1ac7c6 to self. 2023-02-08 08:53:51,514 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed a6defd68663718a57a591a97ac1ac7c6 2023-02-08 08:53:51,515 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=a6defd68663718a57a591a97ac1ac7c6, regionState=CLOSED 2023-02-08 08:53:51,515 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675846431515"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846431515"}]},"ts":"1675846431515"} 2023-02-08 08:53:51,519 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-02-08 08:53:51,519 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure a6defd68663718a57a591a97ac1ac7c6, server=jenkins-hbase4.apache.org,42239,1675846413348 in 682 msec 2023-02-08 08:53:51,520 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=a6defd68663718a57a591a97ac1ac7c6, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=true 2023-02-08 08:53:51,670 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=a6defd68663718a57a591a97ac1ac7c6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:51,671 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675846431670"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846431670"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846431670"}]},"ts":"1675846431670"} 2023-02-08 08:53:51,673 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure a6defd68663718a57a591a97ac1ac7c6, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:53:51,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:51,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a6defd68663718a57a591a97ac1ac7c6, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:53:51,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly a6defd68663718a57a591a97ac1ac7c6 2023-02-08 08:53:51,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:53:51,830 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a6defd68663718a57a591a97ac1ac7c6 2023-02-08 08:53:51,830 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a6defd68663718a57a591a97ac1ac7c6 2023-02-08 08:53:51,831 INFO [StoreOpener-a6defd68663718a57a591a97ac1ac7c6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a6defd68663718a57a591a97ac1ac7c6 2023-02-08 08:53:51,832 DEBUG [StoreOpener-a6defd68663718a57a591a97ac1ac7c6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1 2023-02-08 08:53:51,832 DEBUG [StoreOpener-a6defd68663718a57a591a97ac1ac7c6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1 2023-02-08 08:53:51,833 INFO [StoreOpener-a6defd68663718a57a591a97ac1ac7c6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a6defd68663718a57a591a97ac1ac7c6 columnFamilyName family1 2023-02-08 08:53:51,840 DEBUG [StoreOpener-a6defd68663718a57a591a97ac1ac7c6-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/5deba825d9c64eb2b1703ad565fc6768 2023-02-08 08:53:51,841 INFO [StoreOpener-a6defd68663718a57a591a97ac1ac7c6-1] regionserver.HStore(310): Store=a6defd68663718a57a591a97ac1ac7c6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:53:51,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6 2023-02-08 08:53:51,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6 2023-02-08 08:53:51,846 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a6defd68663718a57a591a97ac1ac7c6 2023-02-08 08:53:51,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a6defd68663718a57a591a97ac1ac7c6; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74889652, jitterRate=0.11594277620315552}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 08:53:51,847 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a6defd68663718a57a591a97ac1ac7c6: Running coprocessor pre-open hook at 1675846431830Writing region info on filesystem at 1675846431830Initializing all the Stores at 1675846431830Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675846431831 (+1 ms)Cleaning up temporary data from old regions at 1675846431843 (+12 ms)Cleaning up detritus from prior splits at 1675846431844 (+1 ms)Running coprocessor post-open hooks at 1675846431846 (+2 ms)Region opened successfully at 1675846431846 2023-02-08 08:53:51,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6., pid=17, masterSystemTime=1675846431825 2023-02-08 08:53:51,849 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:51,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:53:51,850 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=a6defd68663718a57a591a97ac1ac7c6, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:51,850 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1675846431850"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846431850"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846431850"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846431850"}]},"ts":"1675846431850"} 2023-02-08 08:53:51,854 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-08 08:53:51,854 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure a6defd68663718a57a591a97ac1ac7c6, server=jenkins-hbase4.apache.org,42239,1675846413348 in 179 msec 2023-02-08 08:53:51,856 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-02-08 08:53:51,856 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=a6defd68663718a57a591a97ac1ac7c6, REOPEN/MOVE in 1.0240 sec 2023-02-08 08:53:51,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-08 08:53:51,859 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.0400 sec 2023-02-08 08:53:51,861 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.4720 sec 2023-02-08 08:53:51,863 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:51,864 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:51,864 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-02-08 08:53:51,865 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:51,866 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:51,870 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:51,870 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:51,871 INFO [Listener at localhost/40763] compactions.TestMobCompactor(278): alter status finished 2023-02-08 08:53:51,872 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37869] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-08 08:53:51,874 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-08 08:53:51,874 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-08 08:53:51,875 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:53:51,875 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675847031875 2023-02-08 08:53:51,876 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 08:53:51,877 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-02-08 08:53:51,877 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-08 08:53:51,877 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] zookeeper.ReadOnlyZKClient(139): Connect 0x1ffab1ec to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:53:51,885 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1713837b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:53:51,886 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 08:53:51,886 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 08:53:51,886 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 08:53:51,886 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 08:53:51,886 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 08:53:52,722 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128a2ed26dbe2e947009aaab680b90c6692 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128a2ed26dbe2e947009aaab680b90c6692 2023-02-08 08:53:52,723 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-08 08:53:52,736 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:53:52,737 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43388, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:53:52,743 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:52,744 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:52,744 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 08:53:52,745 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48120, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 08:53:52,756 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/85c9261e7c9c424ca44de7494e87309d first=Optional[r05] last=Optional[r8] 2023-02-08 08:53:52,756 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=51 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/85c9261e7c9c424ca44de7494e87309d}] 2023-02-08 08:53:52,761 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/85c9261e7c9c424ca44de7494e87309d for inclusion in a6defd68663718a57a591a97ac1ac7c6/family1 2023-02-08 08:53:52,764 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-08 08:53:52,764 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 08:53:52,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for a6defd68663718a57a591a97ac1ac7c6: 2023-02-08 08:53:52,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/85c9261e7c9c424ca44de7494e87309d to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__13rjn8gfg94kn2ofil6r48djvots9kppksm3sm2fevbqqn309a5r0jgro433nkbg/family1/85c9261e7c9c424ca44de7494e87309d 2023-02-08 08:53:52,768 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__13rjn8gfg94kn2ofil6r48djvots9kppksm3sm2fevbqqn309a5r0jgro433nkbg/family1/85c9261e7c9c424ca44de7494e87309d as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/803cfd814ee94baa93526a5713ee004c_SeqId_53_ 2023-02-08 08:53:52,769 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__13rjn8gfg94kn2ofil6r48djvots9kppksm3sm2fevbqqn309a5r0jgro433nkbg/family1/85c9261e7c9c424ca44de7494e87309d into a6defd68663718a57a591a97ac1ac7c6/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/803cfd814ee94baa93526a5713ee004c_SeqId_53_ - updating store file list. 2023-02-08 08:53:52,774 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/803cfd814ee94baa93526a5713ee004c_SeqId_53_ into a6defd68663718a57a591a97ac1ac7c6/family1 2023-02-08 08:53:52,774 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__13rjn8gfg94kn2ofil6r48djvots9kppksm3sm2fevbqqn309a5r0jgro433nkbg/family1/85c9261e7c9c424ca44de7494e87309d into a6defd68663718a57a591a97ac1ac7c6/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/family1/803cfd814ee94baa93526a5713ee004c_SeqId_53_) 2023-02-08 08:53:52,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__13rjn8gfg94kn2ofil6r48djvots9kppksm3sm2fevbqqn309a5r0jgro433nkbg/family1/85c9261e7c9c424ca44de7494e87309d 2023-02-08 08:53:52,777 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-08 08:53:52,778 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:53:52,779 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511202c95dd3a10ca410fb995ee31fe91b131 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201511202c95dd3a10ca410fb995ee31fe91b131 2023-02-08 08:53:52,781 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128c1a0774ff4a74c59a41b2b8778cf4c5b to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128c1a0774ff4a74c59a41b2b8778cf4c5b 2023-02-08 08:53:52,781 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-02-08 08:53:52,781 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-08 08:53:52,781 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 08:53:52,781 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 08:53:52,781 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-08 08:53:52,781 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-08 08:53:52,781 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:53:52,782 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-08 08:53:52,784 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 909 msec 2023-02-08 08:53:52,865 INFO [Listener at localhost/40763] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=247 (was 209) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:54394 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x10f57606-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:54300 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_922731941_15 at /127.0.0.1:54216 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:54250 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:54342 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:54350 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61594@0x1ffab1ec sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/521398200.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/cluster_d5539413-62cb-b490-c3d3-1723c9563de1/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:42239-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61594@0x77b35877-SendThread(127.0.0.1:61594) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:54388 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61594@0x77b35877-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61594@0x1ffab1ec-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: hconnection-0x10f57606-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:54202 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x44b95804-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:54226 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:54222 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:54316 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:54154 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x10f57606-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/cluster_d5539413-62cb-b490-c3d3-1723c9563de1/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:54298 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61594@0x1ffab1ec-SendThread(127.0.0.1:61594) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61594@0x77b35877 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/521398200.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:54186 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:54196 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x44b95804-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=588 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=217 (was 269), ProcessCount=168 (was 168), AvailableMemoryMB=5070 (was 5190) 2023-02-08 08:53:52,877 INFO [Listener at localhost/40763] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=247, OpenFileDescriptor=588, MaxFileDescriptor=60000, SystemLoadAverage=217, ProcessCount=168, AvailableMemoryMB=5069 2023-02-08 08:53:52,878 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:52,878 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:52,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-08 08:53:52,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:52,882 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:53:52,882 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-02-08 08:53:52,882 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:52,883 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:52,883 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:53:52,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-08 08:53:52,885 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73 2023-02-08 08:53:52,885 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73 empty. 2023-02-08 08:53:52,886 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73 2023-02-08 08:53:52,886 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-02-08 08:53:52,897 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-08 08:53:52,898 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => f453f93eaf555c7a1e29a37805bafe73, NAME => 'testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp 2023-02-08 08:53:52,905 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:53:52,905 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1603): Closing f453f93eaf555c7a1e29a37805bafe73, disabling compactions & flushes 2023-02-08 08:53:52,905 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:52,905 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:52,905 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. after waiting 0 ms 2023-02-08 08:53:52,905 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:52,905 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:52,905 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for f453f93eaf555c7a1e29a37805bafe73: Waiting for close lock at 1675846432905Disabling compacts and flushes for region at 1675846432905Disabling writes for close at 1675846432905Writing region close event to WAL at 1675846432905Closed at 1675846432905 2023-02-08 08:53:52,908 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:53:52,909 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675846432909"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846432909"}]},"ts":"1675846432909"} 2023-02-08 08:53:52,911 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 08:53:52,912 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:53:52,912 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846432912"}]},"ts":"1675846432912"} 2023-02-08 08:53:52,913 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-02-08 08:53:52,917 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=f453f93eaf555c7a1e29a37805bafe73, ASSIGN}] 2023-02-08 08:53:52,918 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=f453f93eaf555c7a1e29a37805bafe73, ASSIGN 2023-02-08 08:53:52,919 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=f453f93eaf555c7a1e29a37805bafe73, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=false 2023-02-08 08:53:52,984 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:52,984 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:52,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-08 08:53:53,070 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=f453f93eaf555c7a1e29a37805bafe73, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,071 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675846433070"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846433070"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846433070"}]},"ts":"1675846433070"} 2023-02-08 08:53:53,073 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure f453f93eaf555c7a1e29a37805bafe73, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:53:53,185 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:53,185 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:53,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-08 08:53:53,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:53,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f453f93eaf555c7a1e29a37805bafe73, NAME => 'testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:53:53,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy f453f93eaf555c7a1e29a37805bafe73 2023-02-08 08:53:53,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:53:53,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f453f93eaf555c7a1e29a37805bafe73 2023-02-08 08:53:53,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f453f93eaf555c7a1e29a37805bafe73 2023-02-08 08:53:53,231 INFO [StoreOpener-f453f93eaf555c7a1e29a37805bafe73-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f453f93eaf555c7a1e29a37805bafe73 2023-02-08 08:53:53,232 DEBUG [StoreOpener-f453f93eaf555c7a1e29a37805bafe73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1 2023-02-08 08:53:53,232 DEBUG [StoreOpener-f453f93eaf555c7a1e29a37805bafe73-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1 2023-02-08 08:53:53,233 INFO [StoreOpener-f453f93eaf555c7a1e29a37805bafe73-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f453f93eaf555c7a1e29a37805bafe73 columnFamilyName family1 2023-02-08 08:53:53,234 INFO [StoreOpener-f453f93eaf555c7a1e29a37805bafe73-1] regionserver.HStore(310): Store=f453f93eaf555c7a1e29a37805bafe73/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:53:53,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73 2023-02-08 08:53:53,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73 2023-02-08 08:53:53,239 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f453f93eaf555c7a1e29a37805bafe73 2023-02-08 08:53:53,241 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:53:53,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f453f93eaf555c7a1e29a37805bafe73; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60576506, jitterRate=-0.09733971953392029}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 08:53:53,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f453f93eaf555c7a1e29a37805bafe73: Running coprocessor pre-open hook at 1675846433229Writing region info on filesystem at 1675846433229Initializing all the Stores at 1675846433230 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675846433230Cleaning up temporary data from old regions at 1675846433237 (+7 ms)Cleaning up detritus from prior splits at 1675846433237Running coprocessor post-open hooks at 1675846433242 (+5 ms)Region opened successfully at 1675846433242 2023-02-08 08:53:53,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73., pid=21, masterSystemTime=1675846433225 2023-02-08 08:53:53,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:53,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:53,245 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=f453f93eaf555c7a1e29a37805bafe73, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,245 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675846433245"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846433245"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846433245"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846433245"}]},"ts":"1675846433245"} 2023-02-08 08:53:53,249 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-02-08 08:53:53,249 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure f453f93eaf555c7a1e29a37805bafe73, server=jenkins-hbase4.apache.org,42239,1675846413348 in 174 msec 2023-02-08 08:53:53,252 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-08 08:53:53,252 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=f453f93eaf555c7a1e29a37805bafe73, ASSIGN in 332 msec 2023-02-08 08:53:53,252 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:53:53,253 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846433253"}]},"ts":"1675846433253"} 2023-02-08 08:53:53,254 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-02-08 08:53:53,258 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:53:53,259 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 379 msec 2023-02-08 08:53:53,486 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:53,487 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:53,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-08 08:53:53,488 INFO [Listener at localhost/40763] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-02-08 08:53:53,518 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:53,518 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:53,519 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:53:53,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,520 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 08:53:53,521 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:53,521 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 08:53:53,521 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 08:53:53,522 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,522 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,523 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:53,523 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:53,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:53,524 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,524 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:53:53,524 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:53:53,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,524 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,525 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,525 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 08:53:53,525 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:53,525 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 08:53:53,525 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:53:53,525 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:53:53,525 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:53,525 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. started... 2023-02-08 08:53:53,526 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f453f93eaf555c7a1e29a37805bafe73 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-08 08:53:53,550 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113028962f2192904f1f96fd4d16afc1b941 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015113028962f2192904f1f96fd4d16afc1b941 2023-02-08 08:53:53,554 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp/family1/9da64b869e414970a7a258009c5a0e28 2023-02-08 08:53:53,560 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp/family1/9da64b869e414970a7a258009c5a0e28 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/9da64b869e414970a7a258009c5a0e28 2023-02-08 08:53:53,566 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/9da64b869e414970a7a258009c5a0e28, entries=1000, sequenceid=5, filesize=166.0 K 2023-02-08 08:53:53,567 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for f453f93eaf555c7a1e29a37805bafe73 in 41ms, sequenceid=5, compaction requested=false 2023-02-08 08:53:53,567 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 08:53:53,568 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f453f93eaf555c7a1e29a37805bafe73: 2023-02-08 08:53:53,568 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:53,569 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:53:53,569 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:53:53,569 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,569 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 08:53:53,569 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-08 08:53:53,571 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,571 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:53,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:53,571 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,571 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:53:53,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:53,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:53,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:53,573 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 08:53:53,573 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4147e7cd[Count = 0] remaining members to acquire global barrier 2023-02-08 08:53:53,573 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 08:53:53,573 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,574 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,575 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 08:53:53,575 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-08 08:53:53,575 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,575 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:53:53,575 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:53:53,578 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 08:53:53,578 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:53,578 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-08 08:53:53,579 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:53,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:53,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:53,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:53,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:53,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:53:53,589 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:53:53,589 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-08 08:53:53,589 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:53:53,589 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:53:53,589 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,589 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:53:53,597 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,597 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:53,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,597 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:53,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:53,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:53,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:53,598 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:53,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:53,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,599 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:53,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,602 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:53,602 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:53,602 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,602 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 08:53:53,602 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:53,602 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,602 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 08:53:53,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:53:53,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:53,602 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:53,603 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,603 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,603 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,604 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,604 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:53:53,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:53,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:53,604 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:53:53,704 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:53:53,704 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:53,704 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:53,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:53:53,709 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:53,710 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:53,710 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:53:53,712 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:53:53,712 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,712 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 08:53:53,712 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:53,713 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 08:53:53,713 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 08:53:53,713 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,713 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,714 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,714 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:53,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:53,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:53,715 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,715 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:53:53,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,715 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:53:53,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,716 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,716 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 08:53:53,716 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,716 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 08:53:53,716 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:53,717 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 08:53:53,717 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:53:53,717 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:53:53,717 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:53,717 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. started... 2023-02-08 08:53:53,717 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f453f93eaf555c7a1e29a37805bafe73 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-08 08:53:53,733 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128d6a689543b954972a29ff25ea4b31aed to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128d6a689543b954972a29ff25ea4b31aed 2023-02-08 08:53:53,739 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp/family1/77d05af2fd4e474396177ee89b43bd68 2023-02-08 08:53:53,745 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp/family1/77d05af2fd4e474396177ee89b43bd68 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/77d05af2fd4e474396177ee89b43bd68 2023-02-08 08:53:53,750 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/77d05af2fd4e474396177ee89b43bd68, entries=1, sequenceid=9, filesize=5.0 K 2023-02-08 08:53:53,751 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for f453f93eaf555c7a1e29a37805bafe73 in 34ms, sequenceid=9, compaction requested=false 2023-02-08 08:53:53,751 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f453f93eaf555c7a1e29a37805bafe73: 2023-02-08 08:53:53,751 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:53,752 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:53:53,752 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:53:53,752 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,752 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 08:53:53,752 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-08 08:53:53,753 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,753 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:53,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:53,754 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,754 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:53:53,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:53,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:53,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:53,756 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 08:53:53,756 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38c89719[Count = 0] remaining members to acquire global barrier 2023-02-08 08:53:53,756 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 08:53:53,756 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,757 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,757 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 08:53:53,757 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-08 08:53:53,757 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:53:53,758 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,758 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:53:53,759 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 08:53:53,759 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,759 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:53,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:53,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:53,760 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-08 08:53:53,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:53,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:53,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:53,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:53:53,763 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:53:53,763 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-08 08:53:53,763 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:53:53,763 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:53:53,763 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,763 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:53:53,766 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,766 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:53,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,766 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:53,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:53,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:53,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:53,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:53,767 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:53,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,773 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:53,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,782 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,783 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:53,783 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,783 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:53,783 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 08:53:53,783 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 08:53:53,783 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:53,783 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:53:53,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:53,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:53,783 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,784 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,784 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,784 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:53:53,784 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,784 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:53,784 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,784 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:53:53,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:53,884 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:53:53,885 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:53,885 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:53,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:53:53,890 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:53,890 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:53,891 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:53:53,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:53:53,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,893 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 08:53:53,893 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:53,894 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 08:53:53,894 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 08:53:53,894 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,894 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,896 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,896 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:53,896 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:53,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:53,896 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:53,897 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:53:53,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,897 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:53:53,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,898 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,898 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 08:53:53,898 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:53,898 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 08:53:53,898 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:53,899 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 08:53:53,899 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:53:53,899 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:53:53,899 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:53,899 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. started... 2023-02-08 08:53:53,899 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f453f93eaf555c7a1e29a37805bafe73 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:53:53,916 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120159326d407e854d23afb9d0eba590f5eb to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120159326d407e854d23afb9d0eba590f5eb 2023-02-08 08:53:54,322 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp/family1/e4d144f514414ffa8be39ea17f9d9ea9 2023-02-08 08:53:54,335 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp/family1/e4d144f514414ffa8be39ea17f9d9ea9 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/e4d144f514414ffa8be39ea17f9d9ea9 2023-02-08 08:53:54,341 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/e4d144f514414ffa8be39ea17f9d9ea9, entries=1, sequenceid=13, filesize=4.9 K 2023-02-08 08:53:54,342 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f453f93eaf555c7a1e29a37805bafe73 in 443ms, sequenceid=13, compaction requested=true 2023-02-08 08:53:54,342 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f453f93eaf555c7a1e29a37805bafe73: 2023-02-08 08:53:54,342 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:54,342 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:53:54,342 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:53:54,342 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:54,342 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 08:53:54,342 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-08 08:53:54,345 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,345 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:54,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:54,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:54,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:54,345 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,346 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:53:54,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:54,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:54,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:54,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:54,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 08:53:54,347 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3cebc895[Count = 0] remaining members to acquire global barrier 2023-02-08 08:53:54,347 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 08:53:54,347 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,348 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,348 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 08:53:54,348 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-08 08:53:54,348 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:53:54,348 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:54,348 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:53:54,349 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:54,349 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 08:53:54,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:54,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:54,350 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:54,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:54,350 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-08 08:53:54,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:54,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:54,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:54,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:54,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:54,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:53:54,352 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:53:54,352 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-08 08:53:54,352 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:53:54,352 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:53:54,352 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,352 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:53:54,354 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,354 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:54,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:54,354 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:54,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,354 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:54,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:54,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:54,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:54,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:54,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,355 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:54,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:54,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:54,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:54,359 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:54,359 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,359 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:54,359 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:54,359 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:54,359 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 08:53:54,359 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:53:54,359 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:54,359 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:54,359 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,359 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:54,360 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:53:54,360 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,360 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:53:54,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:54,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:54,460 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:53:54,461 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:54,461 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:54,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:53:54,466 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:54,466 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:54,467 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:53:54,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:53:54,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,469 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 08:53:54,469 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:54,469 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 08:53:54,469 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 08:53:54,470 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,470 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,471 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:54,471 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:54,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:54,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:54,472 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:54,472 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:53:54,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,472 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:53:54,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,473 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,473 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 08:53:54,473 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:54,473 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 08:53:54,473 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:54,474 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 08:53:54,474 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:53:54,474 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:53:54,474 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:54,474 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. started... 2023-02-08 08:53:54,474 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f453f93eaf555c7a1e29a37805bafe73 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:53:54,892 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120580ba17e748684055ae4c8f047a142147 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120580ba17e748684055ae4c8f047a142147 2023-02-08 08:53:55,298 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp/family1/a54669384dbd496fa65084f11f982451 2023-02-08 08:53:55,306 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp/family1/a54669384dbd496fa65084f11f982451 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/a54669384dbd496fa65084f11f982451 2023-02-08 08:53:55,311 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/a54669384dbd496fa65084f11f982451, entries=1, sequenceid=17, filesize=4.9 K 2023-02-08 08:53:55,312 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f453f93eaf555c7a1e29a37805bafe73 in 838ms, sequenceid=17, compaction requested=true 2023-02-08 08:53:55,312 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f453f93eaf555c7a1e29a37805bafe73: 2023-02-08 08:53:55,312 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:55,312 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:53:55,313 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:53:55,313 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:55,313 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 08:53:55,313 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-08 08:53:55,314 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,314 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:55,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:55,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:55,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:55,315 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,315 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:53:55,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:55,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:55,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:55,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:55,316 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 08:53:55,316 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@528592ed[Count = 0] remaining members to acquire global barrier 2023-02-08 08:53:55,316 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 08:53:55,316 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,318 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,318 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 08:53:55,318 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-08 08:53:55,318 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:55,318 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:53:55,318 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:53:55,320 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:55,320 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 08:53:55,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:55,320 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:55,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:55,320 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-08 08:53:55,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:55,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:55,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:53:55,323 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:53:55,323 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-08 08:53:55,323 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:53:55,323 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:53:55,323 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,323 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:53:55,325 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,325 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,325 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:55,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:55,325 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:55,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:55,325 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:55,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:55,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:55,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:55,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,327 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:55,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:55,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:55,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:55,329 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:55,329 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:55,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:55,329 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:55,329 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 08:53:55,329 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:55,329 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 08:53:55,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:53:55,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:55,329 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,330 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,330 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:55,330 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,330 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,330 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,330 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:53:55,330 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:53:55,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:55,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:55,430 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:53:55,430 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:55,431 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:55,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:53:55,436 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:55,436 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:55,437 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:53:55,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:53:55,438 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,438 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 08:53:55,439 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:55,439 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 08:53:55,439 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 08:53:55,440 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,440 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,441 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:55,441 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:55,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:55,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:55,441 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:55,441 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:53:55,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,442 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:53:55,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,442 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,442 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 08:53:55,442 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:55,443 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 08:53:55,443 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:55,443 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 08:53:55,443 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:53:55,443 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:53:55,443 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:55,443 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. started... 2023-02-08 08:53:55,443 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f453f93eaf555c7a1e29a37805bafe73 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:53:55,860 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228e1ce502751ef4d379518de4ccc3e1825 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228e1ce502751ef4d379518de4ccc3e1825 2023-02-08 08:53:56,266 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp/family1/81caeea50b8947bd8dadd3db2d5e5a63 2023-02-08 08:53:56,272 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp/family1/81caeea50b8947bd8dadd3db2d5e5a63 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/81caeea50b8947bd8dadd3db2d5e5a63 2023-02-08 08:53:56,277 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/81caeea50b8947bd8dadd3db2d5e5a63, entries=1, sequenceid=21, filesize=4.9 K 2023-02-08 08:53:56,278 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f453f93eaf555c7a1e29a37805bafe73 in 835ms, sequenceid=21, compaction requested=true 2023-02-08 08:53:56,279 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f453f93eaf555c7a1e29a37805bafe73: 2023-02-08 08:53:56,279 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:56,279 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:53:56,279 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:53:56,279 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:56,279 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 08:53:56,279 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-08 08:53:56,364 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:56,364 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:56,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:56,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:56,365 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,365 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:53:56,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:56,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:56,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:56,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:56,366 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 08:53:56,366 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2c827eff[Count = 0] remaining members to acquire global barrier 2023-02-08 08:53:56,366 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 08:53:56,366 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,432 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,432 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 08:53:56,432 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-08 08:53:56,432 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:53:56,432 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:56,432 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:53:56,509 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 08:53:56,509 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:56,509 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:56,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:56,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:56,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:56,509 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-08 08:53:56,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:56,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:56,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:56,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:56,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:56,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:53:56,513 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:53:56,513 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-08 08:53:56,513 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:53:56,513 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:53:56,513 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,513 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:53:56,607 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,607 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:56,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:56,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,607 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:56,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:56,608 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:56,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:56,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:56,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:56,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:56,609 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:56,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:56,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:56,672 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:56,672 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:56,672 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,672 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:56,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:56,672 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,672 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 08:53:56,672 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:53:56,672 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:56,672 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:56,672 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:56,673 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,673 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:56,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:56,673 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,673 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:53:56,673 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,674 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:53:56,774 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:53:56,774 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:56,774 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:56,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:53:56,779 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:56,779 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:56,780 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:53:56,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:53:56,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,781 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 08:53:56,782 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:56,782 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 08:53:56,782 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 08:53:56,782 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,783 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,847 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:56,847 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:56,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:56,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:56,848 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:56,848 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:53:56,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,848 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:53:56,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,849 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,849 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 08:53:56,849 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:56,849 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 08:53:56,849 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:56,849 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 08:53:56,850 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:53:56,850 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:53:56,850 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:56,850 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. started... 2023-02-08 08:53:56,850 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f453f93eaf555c7a1e29a37805bafe73 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:53:57,040 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 08:53:57,327 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231d76bb9da3162435fbff8077272fa18a8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231d76bb9da3162435fbff8077272fa18a8 2023-02-08 08:53:57,333 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp/family1/cb74186ab61f4589a5c94645941b0b97 2023-02-08 08:53:57,340 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp/family1/cb74186ab61f4589a5c94645941b0b97 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/cb74186ab61f4589a5c94645941b0b97 2023-02-08 08:53:57,345 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/cb74186ab61f4589a5c94645941b0b97, entries=1, sequenceid=25, filesize=4.9 K 2023-02-08 08:53:57,346 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f453f93eaf555c7a1e29a37805bafe73 in 496ms, sequenceid=25, compaction requested=true 2023-02-08 08:53:57,346 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f453f93eaf555c7a1e29a37805bafe73: 2023-02-08 08:53:57,346 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:57,346 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:53:57,346 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:53:57,346 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:57,347 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 08:53:57,347 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-08 08:53:57,402 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:57,402 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:57,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:57,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:57,402 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:57,402 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:53:57,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:57,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:57,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:57,404 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 08:53:57,404 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@508e4429[Count = 0] remaining members to acquire global barrier 2023-02-08 08:53:57,404 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 08:53:57,404 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,478 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,479 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:57,479 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 08:53:57,479 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-08 08:53:57,479 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:53:57,479 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:53:57,524 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 08:53:57,524 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:57,524 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:57,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:57,525 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-08 08:53:57,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:57,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:57,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:57,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:57,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:57,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:57,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:57,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:53:57,529 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:53:57,529 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-08 08:53:57,529 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:53:57,529 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:53:57,529 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,529 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:53:57,563 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,563 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,564 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:57,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:57,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:57,564 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:57,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:57,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:57,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:57,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:57,566 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:57,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:57,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:57,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:57,617 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:57,617 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:57,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 08:53:57,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:57,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:53:57,617 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:57,617 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:57,617 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,618 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,618 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:57,618 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:57,618 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,618 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:53:57,618 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 08:53:57,619 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:53:57,618 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:57,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:57,719 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:53:57,719 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:57,719 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:57,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:53:57,725 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:57,725 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:57,726 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:53:57,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:53:57,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,728 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 08:53:57,728 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:57,729 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 08:53:57,729 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 08:53:57,729 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,729 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,779 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:57,779 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:57,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:57,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:57,779 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:57,779 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:53:57,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,780 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:53:57,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,780 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,780 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 08:53:57,780 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:57,781 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 08:53:57,781 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:57,781 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 08:53:57,781 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:53:57,782 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:53:57,782 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:57,782 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. started... 2023-02-08 08:53:57,782 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f453f93eaf555c7a1e29a37805bafe73 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:53:58,203 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601014a9ba39c9896409b85a18c45b4c7dd0f to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601014a9ba39c9896409b85a18c45b4c7dd0f 2023-02-08 08:53:58,613 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp/family1/1913070839f44d4486b69c5e9ae9ba5c 2023-02-08 08:53:58,619 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp/family1/1913070839f44d4486b69c5e9ae9ba5c as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/1913070839f44d4486b69c5e9ae9ba5c 2023-02-08 08:53:58,625 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/1913070839f44d4486b69c5e9ae9ba5c, entries=1, sequenceid=29, filesize=4.9 K 2023-02-08 08:53:58,626 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f453f93eaf555c7a1e29a37805bafe73 in 843ms, sequenceid=29, compaction requested=true 2023-02-08 08:53:58,626 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f453f93eaf555c7a1e29a37805bafe73: 2023-02-08 08:53:58,626 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:58,626 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:53:58,626 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:53:58,626 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:58,626 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 08:53:58,626 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-08 08:53:58,678 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:58,678 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:58,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:58,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:58,679 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,679 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:53:58,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:58,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:58,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:58,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:58,680 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 08:53:58,680 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@32c391cd[Count = 0] remaining members to acquire global barrier 2023-02-08 08:53:58,680 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 08:53:58,680 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,724 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,724 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 08:53:58,724 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-08 08:53:58,724 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:53:58,725 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:58,725 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:53:58,755 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 08:53:58,755 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:58,755 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:58,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:58,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:58,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:58,755 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-08 08:53:58,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:58,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:58,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:58,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:58,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:58,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:53:58,758 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:53:58,758 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-08 08:53:58,759 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:53:58,759 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:53:58,759 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,759 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:53:58,801 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,801 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:58,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:58,801 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:58,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:58,802 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:58,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:58,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:58,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:58,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,804 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:58,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:58,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:58,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:58,847 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:58,847 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:58,847 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,848 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:58,848 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,848 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:58,848 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 08:53:58,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:58,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:58,848 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 08:53:58,848 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:53:58,848 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:58,849 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,849 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,849 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:58,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:58,849 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:53:58,849 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:53:58,949 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:53:58,949 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:58,949 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:58,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:53:58,955 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:53:58,955 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:53:58,956 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:53:58,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:53:58,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,958 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 08:53:58,958 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:58,958 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 08:53:58,958 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 08:53:58,959 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:58,959 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,017 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:59,017 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:59,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:59,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:59,017 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:59,017 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:53:59,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,018 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:53:59,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,018 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,018 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 08:53:59,018 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,019 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 08:53:59,019 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:53:59,020 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 08:53:59,020 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:53:59,020 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:53:59,020 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:59,020 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. started... 2023-02-08 08:53:59,020 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f453f93eaf555c7a1e29a37805bafe73 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:53:59,480 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103cfecb91c1abd471689565ecb111ab1bc to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103cfecb91c1abd471689565ecb111ab1bc 2023-02-08 08:53:59,890 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp/family1/9bf952f9e42047eb9fdb68c5d9562b2f 2023-02-08 08:53:59,896 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp/family1/9bf952f9e42047eb9fdb68c5d9562b2f as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/9bf952f9e42047eb9fdb68c5d9562b2f 2023-02-08 08:53:59,900 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/9bf952f9e42047eb9fdb68c5d9562b2f, entries=1, sequenceid=33, filesize=4.9 K 2023-02-08 08:53:59,901 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f453f93eaf555c7a1e29a37805bafe73 in 881ms, sequenceid=33, compaction requested=true 2023-02-08 08:53:59,901 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f453f93eaf555c7a1e29a37805bafe73: 2023-02-08 08:53:59,901 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:53:59,902 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:53:59,902 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:53:59,902 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:59,902 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 08:53:59,902 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-08 08:53:59,903 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:59,903 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:59,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:59,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:59,904 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,904 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:53:59,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:59,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:59,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:59,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:59,905 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 08:53:59,905 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4c2f7f82[Count = 0] remaining members to acquire global barrier 2023-02-08 08:53:59,905 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 08:53:59,905 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,907 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,907 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 08:53:59,907 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-08 08:53:59,907 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:53:59,908 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:59,908 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:53:59,911 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 08:53:59,911 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:59,911 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:59,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:59,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:59,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:59,911 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-08 08:53:59,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:59,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:59,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:59,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:59,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:59,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:53:59,914 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:53:59,914 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-08 08:53:59,914 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:53:59,914 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:53:59,914 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,914 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:53:59,915 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,915 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,915 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:53:59,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,915 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:59,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:53:59,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:59,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:59,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:53:59,916 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:59,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:53:59,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,917 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:59,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:59,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:53:59,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:59,919 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:59,919 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,919 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:53:59,919 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:53:59,919 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:53:59,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:53:59,920 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,919 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:53:59,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 08:53:59,919 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:53:59,920 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:53:59,920 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,920 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:53:59,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:53:59,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:53:59,921 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:53:59,921 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:00,021 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:00,021 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:00,021 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:00,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:00,026 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:00,026 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:00,027 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:00,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:00,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,029 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 08:54:00,029 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:00,029 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 08:54:00,029 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:00,030 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,030 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,031 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:00,031 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:00,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:00,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:00,031 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:00,031 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:00,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,032 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:00,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,032 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,032 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 08:54:00,032 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,032 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 08:54:00,032 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:00,033 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 08:54:00,033 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:00,033 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:00,033 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:54:00,033 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. started... 2023-02-08 08:54:00,033 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f453f93eaf555c7a1e29a37805bafe73 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:00,450 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907ac1ce5fbf13b442aaca64b0bcd299d4b to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20150907ac1ce5fbf13b442aaca64b0bcd299d4b 2023-02-08 08:54:00,857 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp/family1/11c85ef717394c9e965c068c280d09ec 2023-02-08 08:54:00,864 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp/family1/11c85ef717394c9e965c068c280d09ec as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/11c85ef717394c9e965c068c280d09ec 2023-02-08 08:54:00,869 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/11c85ef717394c9e965c068c280d09ec, entries=1, sequenceid=37, filesize=4.9 K 2023-02-08 08:54:00,870 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f453f93eaf555c7a1e29a37805bafe73 in 837ms, sequenceid=37, compaction requested=true 2023-02-08 08:54:00,871 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f453f93eaf555c7a1e29a37805bafe73: 2023-02-08 08:54:00,871 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:54:00,871 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:00,871 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:00,871 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:00,871 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 08:54:00,871 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-08 08:54:00,872 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,872 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:00,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:00,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:00,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:00,873 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,873 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:00,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:00,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:00,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:00,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:00,874 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 08:54:00,874 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@73e5957e[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:00,874 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:00,874 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,875 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,876 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:00,876 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-08 08:54:00,876 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:00,876 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:00,876 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:00,877 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 08:54:00,877 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:00,877 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:00,877 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-08 08:54:00,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:00,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:00,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:00,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:00,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:00,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:00,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:00,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:00,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:00,881 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:00,881 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-08 08:54:00,881 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:00,881 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:00,881 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,881 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:00,884 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,884 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,884 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:00,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:00,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:00,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:00,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:00,885 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:00,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:00,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:00,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,886 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:00,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:00,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:00,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:00,888 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:00,888 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:00,888 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,888 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,888 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:00,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:00,888 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 08:54:00,888 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:00,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:00,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 08:54:00,888 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:00,888 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:00,888 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,889 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:00,889 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:00,889 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,889 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:00,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:00,990 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:00,990 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:00,990 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:00,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:00,995 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:00,995 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:00,996 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:00,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:00,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,998 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 08:54:00,998 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:00,998 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 08:54:00,998 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:00,999 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:00,999 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,000 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:01,000 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:01,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:01,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:01,000 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:01,000 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:01,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,001 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:01,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,001 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,002 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 08:54:01,002 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,002 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 08:54:01,002 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:01,003 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 08:54:01,003 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:01,003 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:01,003 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:54:01,003 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. started... 2023-02-08 08:54:01,003 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f453f93eaf555c7a1e29a37805bafe73 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:01,423 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120830fceb2b8354ca1ba6303428f491a4a to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120830fceb2b8354ca1ba6303428f491a4a 2023-02-08 08:54:01,830 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp/family1/01a515df731c452bb7517893c2eb8643 2023-02-08 08:54:01,837 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp/family1/01a515df731c452bb7517893c2eb8643 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/01a515df731c452bb7517893c2eb8643 2023-02-08 08:54:01,842 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/01a515df731c452bb7517893c2eb8643, entries=1, sequenceid=41, filesize=4.9 K 2023-02-08 08:54:01,843 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for f453f93eaf555c7a1e29a37805bafe73 in 840ms, sequenceid=41, compaction requested=true 2023-02-08 08:54:01,843 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f453f93eaf555c7a1e29a37805bafe73: 2023-02-08 08:54:01,844 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:54:01,844 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:01,844 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:01,844 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:01,844 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 08:54:01,844 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-08 08:54:01,848 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:01,848 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:01,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:01,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:01,848 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,848 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:01,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:01,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:01,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:01,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:01,849 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 08:54:01,849 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@51e83827[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:01,849 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:01,850 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,851 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,851 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:01,851 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-08 08:54:01,851 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:01,851 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:01,851 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:01,852 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:01,852 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 08:54:01,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:01,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:01,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:01,852 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:01,853 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-08 08:54:01,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:01,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:01,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:01,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:01,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:01,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:01,855 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:01,855 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-08 08:54:01,856 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:01,856 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:01,856 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,856 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:01,857 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,857 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,857 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:01,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:01,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:01,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:01,858 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:01,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:01,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:01,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:01,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,859 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:01,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:01,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:01,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,867 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:01,867 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:01,867 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:01,868 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-08 08:54:01,868 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:01,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:01,868 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:01,868 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:01,868 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,868 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 08:54:01,869 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:01,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:01,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:01,868 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:01,870 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:01,870 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,870 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,870 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,970 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:01,970 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:01,970 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:01,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:01,973 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37869] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-02-08 08:54:01,975 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 08:54:01,975 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 08:54:01,976 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:54:01,976 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675847041976 2023-02-08 08:54:01,978 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 08:54:01,979 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-02-08 08:54:01,979 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 08:54:01,980 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] zookeeper.ReadOnlyZKClient(139): Connect 0x26a096c8 to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:54:01,992 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2da3d13c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:54:01,992 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 08:54:02,058 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103081fb880780048ebbb3b0e522f8a1124 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103081fb880780048ebbb3b0e522f8a1124 2023-02-08 08:54:02,059 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 08:54:02,077 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:54:02,078 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50236, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:54:02,086 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:02,086 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:02,087 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 08:54:02,088 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45882, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 08:54:02,101 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/bb5c438f89b54559b4ae56a7b962103b first=Optional[r5] last=Optional[r6] 2023-02-08 08:54:02,102 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/bb5c438f89b54559b4ae56a7b962103b}] 2023-02-08 08:54:02,107 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/bb5c438f89b54559b4ae56a7b962103b for inclusion in f453f93eaf555c7a1e29a37805bafe73/family1 2023-02-08 08:54:02,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-02-08 08:54:02,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 08:54:02,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for f453f93eaf555c7a1e29a37805bafe73: 2023-02-08 08:54:02,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/bb5c438f89b54559b4ae56a7b962103b to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithMonthlyPolicy__anqudothe4l8jp81latu8qo7nf7qdsi8nardjbp2pun5f8p49f8j5clvm4g4e61a/family1/bb5c438f89b54559b4ae56a7b962103b 2023-02-08 08:54:02,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithMonthlyPolicy__anqudothe4l8jp81latu8qo7nf7qdsi8nardjbp2pun5f8p49f8j5clvm4g4e61a/family1/bb5c438f89b54559b4ae56a7b962103b as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/5ffc6f1d6ab1426d97b14f618d620d4b_SeqId_44_ 2023-02-08 08:54:02,117 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithMonthlyPolicy__anqudothe4l8jp81latu8qo7nf7qdsi8nardjbp2pun5f8p49f8j5clvm4g4e61a/family1/bb5c438f89b54559b4ae56a7b962103b into f453f93eaf555c7a1e29a37805bafe73/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/5ffc6f1d6ab1426d97b14f618d620d4b_SeqId_44_ - updating store file list. 2023-02-08 08:54:02,123 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/5ffc6f1d6ab1426d97b14f618d620d4b_SeqId_44_ into f453f93eaf555c7a1e29a37805bafe73/family1 2023-02-08 08:54:02,123 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithMonthlyPolicy__anqudothe4l8jp81latu8qo7nf7qdsi8nardjbp2pun5f8p49f8j5clvm4g4e61a/family1/bb5c438f89b54559b4ae56a7b962103b into f453f93eaf555c7a1e29a37805bafe73/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/5ffc6f1d6ab1426d97b14f618d620d4b_SeqId_44_) 2023-02-08 08:54:02,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithMonthlyPolicy__anqudothe4l8jp81latu8qo7nf7qdsi8nardjbp2pun5f8p49f8j5clvm4g4e61a/family1/bb5c438f89b54559b4ae56a7b962103b 2023-02-08 08:54:02,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-08 08:54:02,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-08 08:54:02,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(1912): f453f93eaf555c7a1e29a37805bafe73/family1 is initiating minor compaction 2023-02-08 08:54:02,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6496822; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:02,129 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f453f93eaf555c7a1e29a37805bafe73/family1 in testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:54:02,129 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/77d05af2fd4e474396177ee89b43bd68, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/e4d144f514414ffa8be39ea17f9d9ea9, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/a54669384dbd496fa65084f11f982451, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/81caeea50b8947bd8dadd3db2d5e5a63, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/cb74186ab61f4589a5c94645941b0b97, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/1913070839f44d4486b69c5e9ae9ba5c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/9bf952f9e42047eb9fdb68c5d9562b2f, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/11c85ef717394c9e965c068c280d09ec, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/01a515df731c452bb7517893c2eb8643, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/5ffc6f1d6ab1426d97b14f618d620d4b_SeqId_44_] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp, totalSize=49.7 K 2023-02-08 08:54:02,129 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 77d05af2fd4e474396177ee89b43bd68, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-08 08:54:02,129 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting e4d144f514414ffa8be39ea17f9d9ea9, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-08 08:54:02,129 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting a54669384dbd496fa65084f11f982451, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-08 08:54:02,129 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 81caeea50b8947bd8dadd3db2d5e5a63, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-08 08:54:02,130 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting cb74186ab61f4589a5c94645941b0b97, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-08 08:54:02,130 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 1913070839f44d4486b69c5e9ae9ba5c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-08 08:54:02,130 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 9bf952f9e42047eb9fdb68c5d9562b2f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-08 08:54:02,130 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 11c85ef717394c9e965c068c280d09ec, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-08 08:54:02,130 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 01a515df731c452bb7517893c2eb8643, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-08 08:54:02,130 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 5ffc6f1d6ab1426d97b14f618d620d4b_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-08 08:54:02,131 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 08:54:02,132 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:02,134 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601014a9ba39c9896409b85a18c45b4c7dd0f to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201601014a9ba39c9896409b85a18c45b4c7dd0f 2023-02-08 08:54:02,136 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103cfecb91c1abd471689565ecb111ab1bc to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103cfecb91c1abd471689565ecb111ab1bc 2023-02-08 08:54:02,136 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-02-08 08:54:02,136 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-02-08 08:54:02,136 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 08:54:02,136 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 08:54:02,210 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] throttle.PressureAwareThroughputController(145): f453f93eaf555c7a1e29a37805bafe73#family1#compaction#22 average throughput is 0.27 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:02,264 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512314e528970a3434c4bb22ec0c481ec1298 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512314e528970a3434c4bb22ec0c481ec1298 2023-02-08 08:54:02,265 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 08:54:02,288 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:02,289 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:02,297 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/c6057c6c9b664e2db098571f3b3afbd7 first=Optional[r1] last=Optional[r4] 2023-02-08 08:54:02,298 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/c6057c6c9b664e2db098571f3b3afbd7}] 2023-02-08 08:54:02,300 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/c6057c6c9b664e2db098571f3b3afbd7 for inclusion in f453f93eaf555c7a1e29a37805bafe73/family1 2023-02-08 08:54:02,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-02-08 08:54:02,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 08:54:02,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for f453f93eaf555c7a1e29a37805bafe73: 2023-02-08 08:54:02,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/c6057c6c9b664e2db098571f3b3afbd7 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithMonthlyPolicy__1n3dn9lo3lhdotpofquqt7mhf1rej4l19h7d8v7he4qlkfqa8momlqaqmnuplmjf/family1/c6057c6c9b664e2db098571f3b3afbd7 2023-02-08 08:54:02,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithMonthlyPolicy__1n3dn9lo3lhdotpofquqt7mhf1rej4l19h7d8v7he4qlkfqa8momlqaqmnuplmjf/family1/c6057c6c9b664e2db098571f3b3afbd7 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/e4fa6f487b964ec7a773f663684e7154_SeqId_46_ 2023-02-08 08:54:02,309 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithMonthlyPolicy__1n3dn9lo3lhdotpofquqt7mhf1rej4l19h7d8v7he4qlkfqa8momlqaqmnuplmjf/family1/c6057c6c9b664e2db098571f3b3afbd7 into f453f93eaf555c7a1e29a37805bafe73/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/e4fa6f487b964ec7a773f663684e7154_SeqId_46_ - updating store file list. 2023-02-08 08:54:02,314 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/e4fa6f487b964ec7a773f663684e7154_SeqId_46_ into f453f93eaf555c7a1e29a37805bafe73/family1 2023-02-08 08:54:02,314 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithMonthlyPolicy__1n3dn9lo3lhdotpofquqt7mhf1rej4l19h7d8v7he4qlkfqa8momlqaqmnuplmjf/family1/c6057c6c9b664e2db098571f3b3afbd7 into f453f93eaf555c7a1e29a37805bafe73/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/e4fa6f487b964ec7a773f663684e7154_SeqId_46_) 2023-02-08 08:54:02,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithMonthlyPolicy__1n3dn9lo3lhdotpofquqt7mhf1rej4l19h7d8v7he4qlkfqa8momlqaqmnuplmjf/family1/c6057c6c9b664e2db098571f3b3afbd7 2023-02-08 08:54:02,317 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 08:54:02,317 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:02,319 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120159326d407e854d23afb9d0eba590f5eb to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120159326d407e854d23afb9d0eba590f5eb 2023-02-08 08:54:02,321 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120580ba17e748684055ae4c8f047a142147 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015120580ba17e748684055ae4c8f047a142147 2023-02-08 08:54:02,322 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228e1ce502751ef4d379518de4ccc3e1825 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151228e1ce502751ef4d379518de4ccc3e1825 2023-02-08 08:54:02,323 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231d76bb9da3162435fbff8077272fa18a8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231d76bb9da3162435fbff8077272fa18a8 2023-02-08 08:54:02,323 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-02-08 08:54:02,323 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-02-08 08:54:02,324 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 08:54:02,324 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 08:54:02,377 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015113070a8fd79493d497cbe6e9c6d65c832d1 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015113070a8fd79493d497cbe6e9c6d65c832d1 2023-02-08 08:54:02,378 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 08:54:02,400 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:02,400 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:02,414 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/e407391016de434a8a2baa5ea086654b first=Optional[r00] last=Optional[r8] 2023-02-08 08:54:02,415 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/e407391016de434a8a2baa5ea086654b}] 2023-02-08 08:54:02,418 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/e407391016de434a8a2baa5ea086654b for inclusion in f453f93eaf555c7a1e29a37805bafe73/family1 2023-02-08 08:54:02,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-02-08 08:54:02,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 08:54:02,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for f453f93eaf555c7a1e29a37805bafe73: 2023-02-08 08:54:02,423 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/e407391016de434a8a2baa5ea086654b to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithMonthlyPolicy__raeajk9o27hlo6ug1d3d99pgm9ra06qifcqsdo63t34cj13kjoft165b609tkirq/family1/e407391016de434a8a2baa5ea086654b 2023-02-08 08:54:02,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithMonthlyPolicy__raeajk9o27hlo6ug1d3d99pgm9ra06qifcqsdo63t34cj13kjoft165b609tkirq/family1/e407391016de434a8a2baa5ea086654b as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/fc6cc9f2461245d9bc02cc7eac253916_SeqId_48_ 2023-02-08 08:54:02,428 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithMonthlyPolicy__raeajk9o27hlo6ug1d3d99pgm9ra06qifcqsdo63t34cj13kjoft165b609tkirq/family1/e407391016de434a8a2baa5ea086654b into f453f93eaf555c7a1e29a37805bafe73/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/fc6cc9f2461245d9bc02cc7eac253916_SeqId_48_ - updating store file list. 2023-02-08 08:54:02,433 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/fc6cc9f2461245d9bc02cc7eac253916_SeqId_48_ into f453f93eaf555c7a1e29a37805bafe73/family1 2023-02-08 08:54:02,433 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithMonthlyPolicy__raeajk9o27hlo6ug1d3d99pgm9ra06qifcqsdo63t34cj13kjoft165b609tkirq/family1/e407391016de434a8a2baa5ea086654b into f453f93eaf555c7a1e29a37805bafe73/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/fc6cc9f2461245d9bc02cc7eac253916_SeqId_48_) 2023-02-08 08:54:02,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithMonthlyPolicy__raeajk9o27hlo6ug1d3d99pgm9ra06qifcqsdo63t34cj13kjoft165b609tkirq/family1/e407391016de434a8a2baa5ea086654b 2023-02-08 08:54:02,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.SortedCompactionPolicy(75): Selecting compaction from 13 store files, 10 compacting, 2 eligible, 16 blocking 2023-02-08 08:54:02,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-08 08:54:02,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-08 08:54:02,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.CompactSplit(415): Not compacting testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. because compaction request was cancelled 2023-02-08 08:54:02,438 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 08:54:02,438 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:02,440 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120830fceb2b8354ca1ba6303428f491a4a to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120830fceb2b8354ca1ba6303428f491a4a 2023-02-08 08:54:02,442 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128d6a689543b954972a29ff25ea4b31aed to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151128d6a689543b954972a29ff25ea4b31aed 2023-02-08 08:54:02,443 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015113028962f2192904f1f96fd4d16afc1b941 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015113028962f2192904f1f96fd4d16afc1b941 2023-02-08 08:54:02,443 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-02-08 08:54:02,443 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-08 08:54:02,444 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 08:54:02,444 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 08:54:02,444 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 08:54:02,444 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 08:54:02,444 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:54:02,445 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 08:54:02,447 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 472 msec 2023-02-08 08:54:02,591 INFO [Listener at localhost/40763] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=264 (was 247) Potentially hanging thread: hconnection-0x165dc7c8-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2fd4c29f-shared-pool-22 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x165dc7c8-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:43928 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:43950 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Thread-462 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61594@0x26a096c8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/521398200.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:43960 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:43966 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:43976 [Waiting for operation #37] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:43980 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x165dc7c8-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x165dc7c8-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61594@0x26a096c8-SendThread(127.0.0.1:61594) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:43904 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:43852 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x165dc7c8-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:43842 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:43944 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:43830 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:44056 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61594@0x26a096c8-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:44006 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:43932 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:43908 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:43860 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:54316 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2fd4c29f-shared-pool-23 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x165dc7c8-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x165dc7c8-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:43786 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:43992 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103a455240433394c8390a62080e14515d4 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:44046 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:54298 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1706849275_15 at /127.0.0.1:43922 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=630 (was 588) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=231 (was 217) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=4726 (was 5069) 2023-02-08 08:54:02,606 INFO [Listener at localhost/40763] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=264, OpenFileDescriptor=630, MaxFileDescriptor=60000, SystemLoadAverage=231, ProcessCount=168, AvailableMemoryMB=4725 2023-02-08 08:54:02,607 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:02,607 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:02,609 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-08 08:54:02,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:02,617 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:54:02,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-02-08 08:54:02,617 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:02,617 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:02,618 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:54:02,618 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-08 08:54:02,624 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5 2023-02-08 08:54:02,625 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5 empty. 2023-02-08 08:54:02,626 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5 2023-02-08 08:54:02,626 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-02-08 08:54:02,667 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-08 08:54:02,669 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 4b057d76430e4231e80be3c284b60af5, NAME => 'testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp 2023-02-08 08:54:02,698 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:02,698 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1603): Closing 4b057d76430e4231e80be3c284b60af5, disabling compactions & flushes 2023-02-08 08:54:02,698 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:02,698 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:02,698 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. after waiting 0 ms 2023-02-08 08:54:02,698 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:02,698 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1837): Closed testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:02,698 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 4b057d76430e4231e80be3c284b60af5: Waiting for close lock at 1675846442698Disabling compacts and flushes for region at 1675846442698Disabling writes for close at 1675846442698Writing region close event to WAL at 1675846442698Closed at 1675846442698 2023-02-08 08:54:02,702 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:54:02,703 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675846442703"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846442703"}]},"ts":"1675846442703"} 2023-02-08 08:54:02,706 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 08:54:02,707 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:54:02,707 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846442707"}]},"ts":"1675846442707"} 2023-02-08 08:54:02,709 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-02-08 08:54:02,716 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=4b057d76430e4231e80be3c284b60af5, ASSIGN}] 2023-02-08 08:54:02,719 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:02,719 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:02,719 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=4b057d76430e4231e80be3c284b60af5, ASSIGN 2023-02-08 08:54:02,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-08 08:54:02,720 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=4b057d76430e4231e80be3c284b60af5, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=false 2023-02-08 08:54:02,872 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=4b057d76430e4231e80be3c284b60af5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:02,872 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675846442872"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846442872"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846442872"}]},"ts":"1675846442872"} 2023-02-08 08:54:02,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure 4b057d76430e4231e80be3c284b60af5, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:54:02,920 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:02,921 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:02,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-08 08:54:03,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:03,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4b057d76430e4231e80be3c284b60af5, NAME => 'testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:54:03,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy 4b057d76430e4231e80be3c284b60af5 2023-02-08 08:54:03,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:03,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 4b057d76430e4231e80be3c284b60af5 2023-02-08 08:54:03,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 4b057d76430e4231e80be3c284b60af5 2023-02-08 08:54:03,036 INFO [StoreOpener-4b057d76430e4231e80be3c284b60af5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 4b057d76430e4231e80be3c284b60af5 2023-02-08 08:54:03,037 DEBUG [StoreOpener-4b057d76430e4231e80be3c284b60af5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1 2023-02-08 08:54:03,037 DEBUG [StoreOpener-4b057d76430e4231e80be3c284b60af5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1 2023-02-08 08:54:03,038 INFO [StoreOpener-4b057d76430e4231e80be3c284b60af5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4b057d76430e4231e80be3c284b60af5 columnFamilyName family1 2023-02-08 08:54:03,039 INFO [StoreOpener-4b057d76430e4231e80be3c284b60af5-1] regionserver.HStore(310): Store=4b057d76430e4231e80be3c284b60af5/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:03,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5 2023-02-08 08:54:03,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5 2023-02-08 08:54:03,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 4b057d76430e4231e80be3c284b60af5 2023-02-08 08:54:03,045 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:54:03,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 4b057d76430e4231e80be3c284b60af5; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70181834, jitterRate=0.04579082131385803}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 08:54:03,046 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 4b057d76430e4231e80be3c284b60af5: Running coprocessor pre-open hook at 1675846443034Writing region info on filesystem at 1675846443034Initializing all the Stores at 1675846443035 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675846443035Cleaning up temporary data from old regions at 1675846443041 (+6 ms)Cleaning up detritus from prior splits at 1675846443041Running coprocessor post-open hooks at 1675846443046 (+5 ms)Region opened successfully at 1675846443046 2023-02-08 08:54:03,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5., pid=25, masterSystemTime=1675846443030 2023-02-08 08:54:03,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:03,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:03,049 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=4b057d76430e4231e80be3c284b60af5, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:03,049 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675846443049"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846443049"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846443049"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846443049"}]},"ts":"1675846443049"} 2023-02-08 08:54:03,052 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-02-08 08:54:03,052 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure 4b057d76430e4231e80be3c284b60af5, server=jenkins-hbase4.apache.org,42239,1675846413348 in 172 msec 2023-02-08 08:54:03,054 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-02-08 08:54:03,054 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=4b057d76430e4231e80be3c284b60af5, ASSIGN in 336 msec 2023-02-08 08:54:03,055 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:54:03,055 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846443055"}]},"ts":"1675846443055"} 2023-02-08 08:54:03,056 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-02-08 08:54:03,059 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:54:03,061 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 450 msec 2023-02-08 08:54:03,074 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/.tmp/family1/e687290cffde42679d7d0f4a781d20d6 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/e687290cffde42679d7d0f4a781d20d6 2023-02-08 08:54:03,081 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in f453f93eaf555c7a1e29a37805bafe73/family1 of f453f93eaf555c7a1e29a37805bafe73 into e687290cffde42679d7d0f4a781d20d6(size=6.6 K), total size for store is 344.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:03,081 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f453f93eaf555c7a1e29a37805bafe73: 2023-02-08 08:54:03,081 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1675846442128; duration=0sec 2023-02-08 08:54:03,082 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:03,135 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/77d05af2fd4e474396177ee89b43bd68, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/e4d144f514414ffa8be39ea17f9d9ea9, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/a54669384dbd496fa65084f11f982451, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/81caeea50b8947bd8dadd3db2d5e5a63, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/cb74186ab61f4589a5c94645941b0b97, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/1913070839f44d4486b69c5e9ae9ba5c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/9bf952f9e42047eb9fdb68c5d9562b2f, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/11c85ef717394c9e965c068c280d09ec, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/01a515df731c452bb7517893c2eb8643, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/5ffc6f1d6ab1426d97b14f618d620d4b_SeqId_44_] to archive 2023-02-08 08:54:03,136 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:03,138 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/77d05af2fd4e474396177ee89b43bd68 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/77d05af2fd4e474396177ee89b43bd68 2023-02-08 08:54:03,140 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/e4d144f514414ffa8be39ea17f9d9ea9 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/e4d144f514414ffa8be39ea17f9d9ea9 2023-02-08 08:54:03,141 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/a54669384dbd496fa65084f11f982451 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/a54669384dbd496fa65084f11f982451 2023-02-08 08:54:03,143 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/81caeea50b8947bd8dadd3db2d5e5a63 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/81caeea50b8947bd8dadd3db2d5e5a63 2023-02-08 08:54:03,144 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/cb74186ab61f4589a5c94645941b0b97 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/cb74186ab61f4589a5c94645941b0b97 2023-02-08 08:54:03,145 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/1913070839f44d4486b69c5e9ae9ba5c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/1913070839f44d4486b69c5e9ae9ba5c 2023-02-08 08:54:03,147 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/9bf952f9e42047eb9fdb68c5d9562b2f to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/9bf952f9e42047eb9fdb68c5d9562b2f 2023-02-08 08:54:03,148 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/11c85ef717394c9e965c068c280d09ec to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/11c85ef717394c9e965c068c280d09ec 2023-02-08 08:54:03,149 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/01a515df731c452bb7517893c2eb8643 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/01a515df731c452bb7517893c2eb8643 2023-02-08 08:54:03,150 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/5ffc6f1d6ab1426d97b14f618d620d4b_SeqId_44_ to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/family1/5ffc6f1d6ab1426d97b14f618d620d4b_SeqId_44_ 2023-02-08 08:54:03,222 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:03,223 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:03,223 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-08 08:54:03,224 INFO [Listener at localhost/40763] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-02-08 08:54:03,249 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:03,249 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:03,250 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:03,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,252 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 08:54:03,252 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:03,252 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 08:54:03,252 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:03,253 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,253 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,254 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:03,254 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:03,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:03,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:03,254 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:03,254 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:03,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,255 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:03,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,255 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,255 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,255 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 08:54:03,255 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:03,255 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 08:54:03,256 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:03,256 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:03,256 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:03,256 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. started... 2023-02-08 08:54:03,256 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4b057d76430e4231e80be3c284b60af5 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-08 08:54:03,278 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511307116a304f9d0477faf592d95f1b5683b to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511307116a304f9d0477faf592d95f1b5683b 2023-02-08 08:54:03,682 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/2636cf1ce76145ebb7d9109daa8a4df2 2023-02-08 08:54:03,689 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/2636cf1ce76145ebb7d9109daa8a4df2 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/2636cf1ce76145ebb7d9109daa8a4df2 2023-02-08 08:54:03,694 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/2636cf1ce76145ebb7d9109daa8a4df2, entries=1000, sequenceid=5, filesize=165.1 K 2023-02-08 08:54:03,695 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 4b057d76430e4231e80be3c284b60af5 in 439ms, sequenceid=5, compaction requested=false 2023-02-08 08:54:03,695 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 08:54:03,696 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4b057d76430e4231e80be3c284b60af5: 2023-02-08 08:54:03,696 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:03,696 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:03,696 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:03,696 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:03,696 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 08:54:03,696 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-08 08:54:03,698 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:03,698 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:03,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:03,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:03,698 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,698 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:03,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:03,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:03,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:03,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:03,700 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 08:54:03,700 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7f8b962e[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:03,700 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:03,700 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,702 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,702 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:03,703 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-08 08:54:03,703 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:03,703 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:03,703 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:03,704 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:03,704 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 08:54:03,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:03,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:03,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:03,704 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:03,704 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-08 08:54:03,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:03,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:03,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:03,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:03,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:03,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:03,707 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:03,707 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-08 08:54:03,707 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:03,707 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:03,707 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,707 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:03,709 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,709 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,709 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:03,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:03,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:03,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,709 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:03,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:03,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:03,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:03,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:03,711 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:03,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:03,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:03,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:03,713 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:03,713 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:03,713 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:03,714 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:03,713 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:03,713 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,714 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 08:54:03,714 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:03,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:03,714 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 08:54:03,714 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:03,714 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,714 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,714 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,714 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:03,714 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:03,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:03,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:03,814 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:03,815 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:03,815 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:03,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:03,820 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:03,820 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:03,820 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:03,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:03,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,822 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 08:54:03,822 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:03,822 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 08:54:03,822 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:03,823 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,823 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,824 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:03,824 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:03,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:03,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:03,825 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:03,825 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:03,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,825 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:03,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,826 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,826 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 08:54:03,826 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:03,826 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 08:54:03,826 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:03,826 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 08:54:03,826 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:03,826 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:03,826 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:03,826 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. started... 2023-02-08 08:54:03,827 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4b057d76430e4231e80be3c284b60af5 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-08 08:54:04,245 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128efa44425debd417c9062f3bfb67687d3 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151128efa44425debd417c9062f3bfb67687d3 2023-02-08 08:54:04,260 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/83959f5e67a249438b970d5b4997d07d 2023-02-08 08:54:04,267 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/83959f5e67a249438b970d5b4997d07d as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/83959f5e67a249438b970d5b4997d07d 2023-02-08 08:54:04,275 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/83959f5e67a249438b970d5b4997d07d, entries=1, sequenceid=9, filesize=5.0 K 2023-02-08 08:54:04,276 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 4b057d76430e4231e80be3c284b60af5 in 449ms, sequenceid=9, compaction requested=false 2023-02-08 08:54:04,277 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4b057d76430e4231e80be3c284b60af5: 2023-02-08 08:54:04,277 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:04,277 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:04,277 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:04,277 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:04,277 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 08:54:04,277 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-08 08:54:04,279 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:04,279 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:04,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:04,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:04,280 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,280 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:04,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:04,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:04,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:04,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:04,282 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 08:54:04,282 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@48ef3532[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:04,282 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:04,282 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,283 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,283 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:04,283 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-08 08:54:04,283 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:04,283 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:04,284 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:04,285 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:04,285 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 08:54:04,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:04,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:04,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:04,285 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:04,285 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-08 08:54:04,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:04,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:04,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:04,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:04,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:04,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:04,288 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:04,288 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-08 08:54:04,288 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:04,288 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:04,288 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,288 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:04,291 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,291 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:04,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:04,291 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:04,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:04,291 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:04,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:04,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:04,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:04,293 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:04,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:04,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:04,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:04,295 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:04,295 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,295 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:04,295 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,295 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:04,295 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:04,295 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 08:54:04,296 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:04,295 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:04,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:04,296 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:04,296 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 08:54:04,296 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,296 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:04,296 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,296 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:04,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:04,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:04,396 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:04,396 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:04,397 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:04,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:04,401 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:04,402 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:04,402 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:04,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:04,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,404 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 08:54:04,404 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:04,405 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 08:54:04,405 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:04,405 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,405 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,408 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:04,408 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:04,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:04,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:04,408 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:04,408 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:04,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,408 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:04,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,409 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,409 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 08:54:04,409 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:04,409 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 08:54:04,409 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:04,409 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 08:54:04,409 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:04,409 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:04,409 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:04,409 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. started... 2023-02-08 08:54:04,410 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4b057d76430e4231e80be3c284b60af5 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:04,825 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120120da319b4963451fa2ecc41250020c29 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120120da319b4963451fa2ecc41250020c29 2023-02-08 08:54:05,231 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/460c23ae7e8d4ff58999ab52750d2896 2023-02-08 08:54:05,238 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/460c23ae7e8d4ff58999ab52750d2896 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/460c23ae7e8d4ff58999ab52750d2896 2023-02-08 08:54:05,243 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/460c23ae7e8d4ff58999ab52750d2896, entries=1, sequenceid=13, filesize=4.9 K 2023-02-08 08:54:05,244 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4b057d76430e4231e80be3c284b60af5 in 834ms, sequenceid=13, compaction requested=true 2023-02-08 08:54:05,245 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4b057d76430e4231e80be3c284b60af5: 2023-02-08 08:54:05,245 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:05,245 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:05,245 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:05,245 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:05,245 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 08:54:05,245 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-08 08:54:05,247 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:05,247 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:05,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:05,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:05,247 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,248 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:05,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:05,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:05,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:05,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:05,249 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 08:54:05,249 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@17fe1734[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:05,249 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:05,249 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,250 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,251 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:05,251 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-08 08:54:05,251 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:05,251 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:05,251 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:05,253 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 08:54:05,253 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:05,253 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:05,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:05,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:05,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:05,253 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-08 08:54:05,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:05,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:05,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:05,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:05,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:05,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:05,257 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:05,257 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-08 08:54:05,258 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:05,258 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:05,258 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,258 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:05,259 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,259 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:05,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:05,260 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:05,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:05,260 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:05,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:05,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:05,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:05,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:05,261 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:05,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:05,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:05,266 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:05,267 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,267 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,267 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,267 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 08:54:05,267 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:05,267 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:05,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,267 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:05,267 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:05,267 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:05,267 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:05,267 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,267 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:05,267 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:05,367 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:05,368 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:05,368 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:05,369 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:05,373 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:05,373 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:05,374 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:05,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:05,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,376 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 08:54:05,376 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:05,376 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 08:54:05,376 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:05,376 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,376 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,377 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:05,377 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:05,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:05,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:05,378 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:05,378 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:05,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,378 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:05,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,378 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,379 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 08:54:05,379 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:05,379 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 08:54:05,379 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:05,379 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 08:54:05,379 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:05,379 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:05,379 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:05,379 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. started... 2023-02-08 08:54:05,379 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4b057d76430e4231e80be3c284b60af5 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:05,794 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205573ae816c9b442e4989714f717a97287 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205573ae816c9b442e4989714f717a97287 2023-02-08 08:54:06,200 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/e6d595d0defa469ca661110ed146a94e 2023-02-08 08:54:06,207 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/e6d595d0defa469ca661110ed146a94e as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/e6d595d0defa469ca661110ed146a94e 2023-02-08 08:54:06,212 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/e6d595d0defa469ca661110ed146a94e, entries=1, sequenceid=17, filesize=4.9 K 2023-02-08 08:54:06,214 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4b057d76430e4231e80be3c284b60af5 in 835ms, sequenceid=17, compaction requested=true 2023-02-08 08:54:06,214 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4b057d76430e4231e80be3c284b60af5: 2023-02-08 08:54:06,214 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:06,214 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:06,214 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:06,214 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:06,214 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 08:54:06,214 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-08 08:54:06,217 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:06,217 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:06,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:06,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:06,217 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,217 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:06,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:06,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:06,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:06,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:06,219 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 08:54:06,219 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7473cec[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:06,219 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:06,219 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,220 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,220 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:06,220 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-08 08:54:06,220 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:06,220 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:06,221 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:06,222 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 08:54:06,222 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:06,222 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:06,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:06,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:06,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:06,222 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-08 08:54:06,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:06,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:06,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:06,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:06,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:06,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:06,225 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:06,225 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-08 08:54:06,225 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:06,225 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:06,225 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,225 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:06,227 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,227 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,227 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:06,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:06,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:06,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:06,227 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:06,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:06,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:06,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:06,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,229 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:06,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:06,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:06,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:06,232 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:06,232 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:06,232 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:06,232 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,232 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:06,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:06,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:06,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:06,233 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:06,233 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,233 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:06,233 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,233 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 08:54:06,233 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 08:54:06,233 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:06,233 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,234 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:06,234 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:06,334 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:06,334 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:06,334 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:06,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:06,339 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:06,339 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:06,340 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:06,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:06,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,341 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 08:54:06,341 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:06,341 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 08:54:06,341 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:06,342 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,342 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,343 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:06,343 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:06,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:06,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:06,343 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:06,343 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:06,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,344 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:06,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,344 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,345 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 08:54:06,345 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:06,345 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 08:54:06,345 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:06,345 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 08:54:06,345 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:06,345 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:06,345 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:06,345 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. started... 2023-02-08 08:54:06,345 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4b057d76430e4231e80be3c284b60af5 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:06,759 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228c5bf3e259df042029be94fa672995fa8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228c5bf3e259df042029be94fa672995fa8 2023-02-08 08:54:07,165 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/8d859ecf5685437794b88da43b39c1ea 2023-02-08 08:54:07,195 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 08:54:07,196 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/8d859ecf5685437794b88da43b39c1ea as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/8d859ecf5685437794b88da43b39c1ea 2023-02-08 08:54:07,196 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x77b35877 to 127.0.0.1:61594 2023-02-08 08:54:07,196 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:54:07,203 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 08:54:07,203 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1ffab1ec to 127.0.0.1:61594 2023-02-08 08:54:07,203 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:54:07,203 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 08:54:07,203 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x26a096c8 to 127.0.0.1:61594 2023-02-08 08:54:07,203 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:54:07,210 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/8d859ecf5685437794b88da43b39c1ea, entries=1, sequenceid=21, filesize=4.9 K 2023-02-08 08:54:07,211 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4b057d76430e4231e80be3c284b60af5 in 866ms, sequenceid=21, compaction requested=true 2023-02-08 08:54:07,211 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4b057d76430e4231e80be3c284b60af5: 2023-02-08 08:54:07,211 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:07,211 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:07,212 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:07,212 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:07,212 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 08:54:07,212 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-08 08:54:07,213 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,213 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:07,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:07,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:07,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:07,214 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:07,214 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:07,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:07,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:07,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:07,215 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 08:54:07,215 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24def0cc[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:07,215 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:07,215 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,216 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,217 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:07,217 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-08 08:54:07,217 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:07,217 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:07,217 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:07,219 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:07,219 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 08:54:07,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:07,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:07,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:07,219 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:07,219 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-08 08:54:07,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:07,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:07,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:07,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:07,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:07,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:07,222 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:07,222 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-08 08:54:07,222 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:07,222 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:07,222 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,222 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:07,224 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,224 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:07,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:07,224 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:07,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,224 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:07,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:07,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:07,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:07,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:07,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,226 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:07,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:07,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:07,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:07,230 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:07,230 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,230 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,230 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:07,230 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,230 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,230 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,230 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:07,230 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:07,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:07,230 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:07,230 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 08:54:07,230 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:07,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:07,230 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:07,231 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:07,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:07,231 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 08:54:07,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:07,331 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:07,331 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:07,331 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:07,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:07,336 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:07,336 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:07,336 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:07,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:07,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,338 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 08:54:07,338 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:07,338 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 08:54:07,339 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:07,339 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,339 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,340 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:07,341 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:07,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:07,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:07,341 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:07,341 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:07,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,341 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:07,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,342 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,342 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 08:54:07,342 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:07,342 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 08:54:07,342 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:07,342 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 08:54:07,342 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:07,342 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:07,342 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:07,342 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. started... 2023-02-08 08:54:07,343 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4b057d76430e4231e80be3c284b60af5 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:07,760 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231adcdd0abe52747fdb6a195dbc7148b7d to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231adcdd0abe52747fdb6a195dbc7148b7d 2023-02-08 08:54:08,183 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/539ae8a87c674194a38fc06820d31381 2023-02-08 08:54:08,188 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/539ae8a87c674194a38fc06820d31381 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/539ae8a87c674194a38fc06820d31381 2023-02-08 08:54:08,193 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/539ae8a87c674194a38fc06820d31381, entries=1, sequenceid=25, filesize=4.9 K 2023-02-08 08:54:08,194 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4b057d76430e4231e80be3c284b60af5 in 851ms, sequenceid=25, compaction requested=true 2023-02-08 08:54:08,194 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4b057d76430e4231e80be3c284b60af5: 2023-02-08 08:54:08,194 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:08,194 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:08,194 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:08,194 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:08,194 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 08:54:08,194 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-08 08:54:08,196 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,196 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:08,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:08,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:08,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:08,196 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,196 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:08,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:08,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:08,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:08,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:08,197 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 08:54:08,197 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:08,197 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15a917c0[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:08,197 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,198 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,199 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:08,199 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-08 08:54:08,199 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:08,199 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:08,199 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:08,203 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 08:54:08,203 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:08,203 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:08,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:08,203 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-08 08:54:08,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:08,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:08,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:08,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:08,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:08,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:08,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:08,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:08,206 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:08,206 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-08 08:54:08,206 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:08,206 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:08,206 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,206 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:08,208 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,208 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,208 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:08,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:08,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:08,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:08,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:08,208 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:08,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:08,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:08,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,209 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:08,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:08,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:08,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:08,211 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:08,211 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,211 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,211 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:08,211 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:08,212 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 08:54:08,212 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:08,212 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:08,211 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:08,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:08,212 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 08:54:08,212 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:08,212 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,212 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:08,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:08,212 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,212 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:08,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:08,312 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:08,312 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:08,313 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:08,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:08,317 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:08,317 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:08,318 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:08,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:08,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,320 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 08:54:08,320 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:08,320 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 08:54:08,320 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:08,320 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,320 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,322 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:08,322 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:08,322 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:08,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:08,322 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:08,322 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:08,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,323 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:08,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,323 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,323 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 08:54:08,323 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:08,324 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 08:54:08,324 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:08,324 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 08:54:08,324 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:08,324 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:08,324 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:08,324 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. started... 2023-02-08 08:54:08,324 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4b057d76430e4231e80be3c284b60af5 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:08,563 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 08:54:08,755 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101ad74bdf852ff4977bfc711004b16ceb0 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101ad74bdf852ff4977bfc711004b16ceb0 2023-02-08 08:54:09,161 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/1bae165846fd4a9fae0d3c381bab8da4 2023-02-08 08:54:09,167 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/1bae165846fd4a9fae0d3c381bab8da4 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/1bae165846fd4a9fae0d3c381bab8da4 2023-02-08 08:54:09,172 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/1bae165846fd4a9fae0d3c381bab8da4, entries=1, sequenceid=29, filesize=4.9 K 2023-02-08 08:54:09,173 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4b057d76430e4231e80be3c284b60af5 in 849ms, sequenceid=29, compaction requested=true 2023-02-08 08:54:09,173 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4b057d76430e4231e80be3c284b60af5: 2023-02-08 08:54:09,173 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:09,173 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:09,174 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:09,174 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,174 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 08:54:09,174 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-08 08:54:09,175 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,175 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:09,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:09,176 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,176 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:09,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:09,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:09,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:09,178 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 08:54:09,178 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7209e5f6[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:09,178 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:09,179 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,180 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,180 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:09,180 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-08 08:54:09,180 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:09,180 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,180 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:09,182 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,182 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 08:54:09,182 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:09,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:09,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:09,182 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-08 08:54:09,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:09,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:09,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:09,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:09,185 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:09,185 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-08 08:54:09,185 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:09,185 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:09,185 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,185 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:09,188 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,188 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:09,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:09,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,188 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:09,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,189 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:09,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,195 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:09,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,198 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,198 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,198 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,198 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:09,198 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 08:54:09,198 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 08:54:09,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:09,198 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,198 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,199 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,199 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,199 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,199 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:09,199 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:09,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,199 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,299 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:09,299 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:09,300 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:09,301 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:09,305 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:09,305 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:09,306 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:09,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:09,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,308 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 08:54:09,308 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:09,308 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 08:54:09,308 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:09,311 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,311 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,313 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,313 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,313 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,313 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:09,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,314 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:09,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,314 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,314 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 08:54:09,315 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,315 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 08:54:09,315 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:09,315 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 08:54:09,315 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:09,315 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:09,315 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:09,315 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. started... 2023-02-08 08:54:09,315 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4b057d76430e4231e80be3c284b60af5 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:09,736 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601038842f864b0814bbaa52d6c3fb9a2f246 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601038842f864b0814bbaa52d6c3fb9a2f246 2023-02-08 08:54:09,744 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/27af5dbe6a0343cb852c58edb43211c4 2023-02-08 08:54:09,750 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/27af5dbe6a0343cb852c58edb43211c4 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/27af5dbe6a0343cb852c58edb43211c4 2023-02-08 08:54:09,757 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/27af5dbe6a0343cb852c58edb43211c4, entries=1, sequenceid=33, filesize=4.9 K 2023-02-08 08:54:09,758 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4b057d76430e4231e80be3c284b60af5 in 443ms, sequenceid=33, compaction requested=true 2023-02-08 08:54:09,758 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4b057d76430e4231e80be3c284b60af5: 2023-02-08 08:54:09,759 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:09,759 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:09,759 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:09,759 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,759 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 08:54:09,759 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-08 08:54:09,766 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,766 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:09,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:09,767 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,767 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:09,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:09,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:09,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:09,769 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 08:54:09,769 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4c170707[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:09,769 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:09,769 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,770 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,770 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:09,770 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-08 08:54:09,770 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:09,770 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,770 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:09,771 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,771 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 08:54:09,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:09,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:09,771 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:09,772 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-08 08:54:09,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:09,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:09,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:09,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:09,774 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:09,774 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-08 08:54:09,775 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:09,775 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:09,775 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,775 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:09,776 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,776 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,776 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:09,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:09,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:09,777 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:09,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,778 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:09,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,782 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,782 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,782 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,782 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,782 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:09,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:09,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:09,782 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:09,782 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 08:54:09,782 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,782 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 08:54:09,782 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:09,782 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,783 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,783 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,783 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:09,783 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:09,883 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:09,883 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:09,883 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:09,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:09,888 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:09,888 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:09,888 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:09,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:09,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,890 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 08:54:09,890 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:09,890 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 08:54:09,890 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:09,890 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,890 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,892 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:09,892 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:09,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:09,892 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:09,892 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:09,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,893 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:09,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,893 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,893 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 08:54:09,893 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:09,893 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 08:54:09,893 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:09,893 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 08:54:09,893 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:09,893 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:09,893 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:09,894 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. started... 2023-02-08 08:54:09,894 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4b057d76430e4231e80be3c284b60af5 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:10,306 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907b2932e0b6fdf4d1fb4794b9aed658034 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20150907b2932e0b6fdf4d1fb4794b9aed658034 2023-02-08 08:54:10,712 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/14a88fba7e1a48a6a2779d25d9b0f27d 2023-02-08 08:54:10,717 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/14a88fba7e1a48a6a2779d25d9b0f27d as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/14a88fba7e1a48a6a2779d25d9b0f27d 2023-02-08 08:54:10,721 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/14a88fba7e1a48a6a2779d25d9b0f27d, entries=1, sequenceid=37, filesize=4.9 K 2023-02-08 08:54:10,723 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4b057d76430e4231e80be3c284b60af5 in 828ms, sequenceid=37, compaction requested=true 2023-02-08 08:54:10,723 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4b057d76430e4231e80be3c284b60af5: 2023-02-08 08:54:10,723 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:10,723 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:10,723 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:10,723 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:10,723 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 08:54:10,723 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-08 08:54:10,725 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:10,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:10,725 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:10,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:10,725 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,725 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:10,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:10,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:10,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:10,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:10,727 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 08:54:10,727 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1f9cc6d9[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:10,727 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:10,727 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,731 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,732 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:10,732 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-08 08:54:10,732 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:10,732 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:10,732 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:10,733 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:10,733 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 08:54:10,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:10,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:10,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:10,733 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:10,734 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-08 08:54:10,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:10,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:10,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:10,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:10,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:10,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:10,736 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:10,736 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-08 08:54:10,737 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:10,737 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:10,737 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,737 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:10,738 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,738 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,739 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:10,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:10,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:10,739 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:10,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:10,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:10,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:10,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:10,740 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:10,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:10,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:10,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:10,747 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:10,747 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,748 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,748 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:10,748 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:10,748 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 08:54:10,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:10,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:10,748 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:10,748 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:10,748 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:10,748 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,748 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,748 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,748 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:10,748 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:10,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:10,748 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 08:54:10,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:10,849 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:10,849 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:10,849 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:10,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:10,853 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:10,853 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:10,854 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:10,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:10,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,855 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 08:54:10,855 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:10,856 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 08:54:10,856 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:10,856 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,856 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,857 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:10,857 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:10,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:10,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:10,858 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:10,858 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:10,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:10,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,859 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,859 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 08:54:10,859 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:10,859 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 08:54:10,859 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:10,859 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 08:54:10,859 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:10,859 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:10,859 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:10,859 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. started... 2023-02-08 08:54:10,860 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4b057d76430e4231e80be3c284b60af5 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:11,273 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511202cdd2aff24b147b198746dd76da37937 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511202cdd2aff24b147b198746dd76da37937 2023-02-08 08:54:11,678 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/f3042eaf33654f808ed07af50167c7a3 2023-02-08 08:54:11,684 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/f3042eaf33654f808ed07af50167c7a3 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/f3042eaf33654f808ed07af50167c7a3 2023-02-08 08:54:11,688 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/f3042eaf33654f808ed07af50167c7a3, entries=1, sequenceid=41, filesize=4.9 K 2023-02-08 08:54:11,689 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 4b057d76430e4231e80be3c284b60af5 in 829ms, sequenceid=41, compaction requested=true 2023-02-08 08:54:11,689 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4b057d76430e4231e80be3c284b60af5: 2023-02-08 08:54:11,689 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:11,689 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:11,689 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:11,689 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:11,689 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 08:54:11,689 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-08 08:54:11,691 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:11,691 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:11,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:11,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:11,691 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,691 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:11,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:11,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:11,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:11,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:11,693 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 08:54:11,693 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@42d6d5fd[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:11,693 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:11,693 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,695 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,695 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:11,695 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-08 08:54:11,695 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:11,695 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:11,695 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:11,696 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 08:54:11,696 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:11,696 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:11,696 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-08 08:54:11,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:11,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:11,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:11,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:11,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:11,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:11,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:11,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:11,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:11,698 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:11,698 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-08 08:54:11,699 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:11,699 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:11,699 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,699 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:11,700 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,700 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,700 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:11,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:11,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:11,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:11,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:11,700 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:11,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:11,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:11,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,701 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:11,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:11,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:11,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:11,703 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:11,703 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:11,703 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:11,703 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-08 08:54:11,703 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:11,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 08:54:11,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:11,703 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:11,703 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,703 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:11,704 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:11,704 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:11,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:11,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:11,704 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,704 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:11,704 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,704 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,804 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:11,804 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:11,804 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:11,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:11,807 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37869] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-02-08 08:54:11,808 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 08:54:11,809 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 08:54:11,810 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:54:11,810 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675847051810 2023-02-08 08:54:11,811 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-08 08:54:11,812 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-02-08 08:54:11,812 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 08:54:11,812 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] zookeeper.ReadOnlyZKClient(139): Connect 0x38aeb583 to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:54:11,820 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@b37b99b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:54:11,821 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 08:54:12,259 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ae0d8e106a8d492ba794f95077dd9ec2 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103ae0d8e106a8d492ba794f95077dd9ec2 2023-02-08 08:54:12,260 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 08:54:12,273 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:54:12,274 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47840, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:54:12,280 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:12,280 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:12,281 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 08:54:12,283 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46802, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 08:54:12,292 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d4decad4843e4a42b6613d44a562bfc4 first=Optional[r3] last=Optional[r6] 2023-02-08 08:54:12,292 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d4decad4843e4a42b6613d44a562bfc4}] 2023-02-08 08:54:12,297 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d4decad4843e4a42b6613d44a562bfc4 for inclusion in 4b057d76430e4231e80be3c284b60af5/family1 2023-02-08 08:54:12,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-08 08:54:12,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 08:54:12,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for 4b057d76430e4231e80be3c284b60af5: 2023-02-08 08:54:12,301 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d4decad4843e4a42b6613d44a562bfc4 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithWeeklyPolicy__iuq4medmc3n0qb5rilis6ngh747i16a98a1qoj4cnsb00r3ic8ncnu3vrktuev2k/family1/d4decad4843e4a42b6613d44a562bfc4 2023-02-08 08:54:12,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithWeeklyPolicy__iuq4medmc3n0qb5rilis6ngh747i16a98a1qoj4cnsb00r3ic8ncnu3vrktuev2k/family1/d4decad4843e4a42b6613d44a562bfc4 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/21e48808703d4f2c9e4c8fc7b21166ee_SeqId_44_ 2023-02-08 08:54:12,304 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithWeeklyPolicy__iuq4medmc3n0qb5rilis6ngh747i16a98a1qoj4cnsb00r3ic8ncnu3vrktuev2k/family1/d4decad4843e4a42b6613d44a562bfc4 into 4b057d76430e4231e80be3c284b60af5/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/21e48808703d4f2c9e4c8fc7b21166ee_SeqId_44_ - updating store file list. 2023-02-08 08:54:12,309 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/21e48808703d4f2c9e4c8fc7b21166ee_SeqId_44_ into 4b057d76430e4231e80be3c284b60af5/family1 2023-02-08 08:54:12,309 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithWeeklyPolicy__iuq4medmc3n0qb5rilis6ngh747i16a98a1qoj4cnsb00r3ic8ncnu3vrktuev2k/family1/d4decad4843e4a42b6613d44a562bfc4 into 4b057d76430e4231e80be3c284b60af5/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/21e48808703d4f2c9e4c8fc7b21166ee_SeqId_44_) 2023-02-08 08:54:12,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithWeeklyPolicy__iuq4medmc3n0qb5rilis6ngh747i16a98a1qoj4cnsb00r3ic8ncnu3vrktuev2k/family1/d4decad4843e4a42b6613d44a562bfc4 2023-02-08 08:54:12,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-08 08:54:12,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-08 08:54:12,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(1912): 4b057d76430e4231e80be3c284b60af5/family1 is initiating minor compaction 2023-02-08 08:54:12,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@53ece138; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:12,314 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 4b057d76430e4231e80be3c284b60af5/family1 in testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:12,314 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/83959f5e67a249438b970d5b4997d07d, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/460c23ae7e8d4ff58999ab52750d2896, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/e6d595d0defa469ca661110ed146a94e, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/8d859ecf5685437794b88da43b39c1ea, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/539ae8a87c674194a38fc06820d31381, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/1bae165846fd4a9fae0d3c381bab8da4, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/27af5dbe6a0343cb852c58edb43211c4, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/14a88fba7e1a48a6a2779d25d9b0f27d, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/f3042eaf33654f808ed07af50167c7a3, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/21e48808703d4f2c9e4c8fc7b21166ee_SeqId_44_] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp, totalSize=50.0 K 2023-02-08 08:54:12,314 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 83959f5e67a249438b970d5b4997d07d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-08 08:54:12,315 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 460c23ae7e8d4ff58999ab52750d2896, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-08 08:54:12,315 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting e6d595d0defa469ca661110ed146a94e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-08 08:54:12,315 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 8d859ecf5685437794b88da43b39c1ea, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-08 08:54:12,315 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 539ae8a87c674194a38fc06820d31381, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-08 08:54:12,315 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 1bae165846fd4a9fae0d3c381bab8da4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-08 08:54:12,315 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 27af5dbe6a0343cb852c58edb43211c4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-08 08:54:12,315 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 14a88fba7e1a48a6a2779d25d9b0f27d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-08 08:54:12,315 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting f3042eaf33654f808ed07af50167c7a3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-08 08:54:12,315 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 21e48808703d4f2c9e4c8fc7b21166ee_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-08 08:54:12,316 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 08:54:12,317 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:12,319 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228c5bf3e259df042029be94fa672995fa8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151228c5bf3e259df042029be94fa672995fa8 2023-02-08 08:54:12,320 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231adcdd0abe52747fdb6a195dbc7148b7d to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231adcdd0abe52747fdb6a195dbc7148b7d 2023-02-08 08:54:12,322 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101ad74bdf852ff4977bfc711004b16ceb0 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160101ad74bdf852ff4977bfc711004b16ceb0 2023-02-08 08:54:12,323 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601038842f864b0814bbaa52d6c3fb9a2f246 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601038842f864b0814bbaa52d6c3fb9a2f246 2023-02-08 08:54:12,323 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-02-08 08:54:12,323 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-08 08:54:12,324 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 08:54:12,324 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 08:54:12,324 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 08:54:12,324 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 08:54:12,324 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 08:54:12,324 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 08:54:12,324 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 08:54:12,324 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 08:54:12,346 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4b057d76430e4231e80be3c284b60af5#family1#compaction#33 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:12,765 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512053c2908b12b9a4825b9fd4662ba23b693 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512053c2908b12b9a4825b9fd4662ba23b693 2023-02-08 08:54:12,766 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 08:54:12,768 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/3577685d47204e728bc84194c2c51941 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/3577685d47204e728bc84194c2c51941 2023-02-08 08:54:12,774 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 4b057d76430e4231e80be3c284b60af5/family1 of 4b057d76430e4231e80be3c284b60af5 into 3577685d47204e728bc84194c2c51941(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:12,774 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4b057d76430e4231e80be3c284b60af5: 2023-02-08 08:54:12,775 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1675846452314; duration=0sec 2023-02-08 08:54:12,775 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:12,786 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:12,786 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:12,795 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7f9e32781bf6461fb16ca3f32672f634 first=Optional[r1] last=Optional[r2] 2023-02-08 08:54:12,796 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7f9e32781bf6461fb16ca3f32672f634}] 2023-02-08 08:54:12,798 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7f9e32781bf6461fb16ca3f32672f634 for inclusion in 4b057d76430e4231e80be3c284b60af5/family1 2023-02-08 08:54:12,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-02-08 08:54:12,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 08:54:12,800 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for 4b057d76430e4231e80be3c284b60af5: 2023-02-08 08:54:12,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/7f9e32781bf6461fb16ca3f32672f634 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithWeeklyPolicy__f9nlm70nkhgrcv529ncsjeanp08uh8ngjkg4hlfehmcjs3254leo9lo7oebbc7iv/family1/7f9e32781bf6461fb16ca3f32672f634 2023-02-08 08:54:12,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithWeeklyPolicy__f9nlm70nkhgrcv529ncsjeanp08uh8ngjkg4hlfehmcjs3254leo9lo7oebbc7iv/family1/7f9e32781bf6461fb16ca3f32672f634 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/7dbcd903ab594ce68b3ade7eb3faeb52_SeqId_47_ 2023-02-08 08:54:12,806 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithWeeklyPolicy__f9nlm70nkhgrcv529ncsjeanp08uh8ngjkg4hlfehmcjs3254leo9lo7oebbc7iv/family1/7f9e32781bf6461fb16ca3f32672f634 into 4b057d76430e4231e80be3c284b60af5/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/7dbcd903ab594ce68b3ade7eb3faeb52_SeqId_47_ - updating store file list. 2023-02-08 08:54:12,810 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/7dbcd903ab594ce68b3ade7eb3faeb52_SeqId_47_ into 4b057d76430e4231e80be3c284b60af5/family1 2023-02-08 08:54:12,810 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithWeeklyPolicy__f9nlm70nkhgrcv529ncsjeanp08uh8ngjkg4hlfehmcjs3254leo9lo7oebbc7iv/family1/7f9e32781bf6461fb16ca3f32672f634 into 4b057d76430e4231e80be3c284b60af5/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/7dbcd903ab594ce68b3ade7eb3faeb52_SeqId_47_) 2023-02-08 08:54:12,810 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithWeeklyPolicy__f9nlm70nkhgrcv529ncsjeanp08uh8ngjkg4hlfehmcjs3254leo9lo7oebbc7iv/family1/7f9e32781bf6461fb16ca3f32672f634 2023-02-08 08:54:12,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-08 08:54:12,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 181014 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-08 08:54:12,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(1912): 4b057d76430e4231e80be3c284b60af5/family1 is initiating minor compaction (all files) 2023-02-08 08:54:12,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2266ca6; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:12,812 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 4b057d76430e4231e80be3c284b60af5/family1 in testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:54:12,812 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/2636cf1ce76145ebb7d9109daa8a4df2, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/3577685d47204e728bc84194c2c51941, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/7dbcd903ab594ce68b3ade7eb3faeb52_SeqId_47_] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp, totalSize=176.8 K 2023-02-08 08:54:12,812 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 2636cf1ce76145ebb7d9109daa8a4df2, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878836535 2023-02-08 08:54:12,813 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 3577685d47204e728bc84194c2c51941, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621236535 2023-02-08 08:54:12,813 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 7dbcd903ab594ce68b3ade7eb3faeb52_SeqId_47_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965236535 2023-02-08 08:54:12,814 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 08:54:12,814 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:12,816 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120120da319b4963451fa2ecc41250020c29 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120120da319b4963451fa2ecc41250020c29 2023-02-08 08:54:12,817 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205573ae816c9b442e4989714f717a97287 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151205573ae816c9b442e4989714f717a97287 2023-02-08 08:54:12,817 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-02-08 08:54:12,817 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-08 08:54:12,818 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 08:54:12,818 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 08:54:12,818 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:54:12,819 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 08:54:12,821 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.0120 sec 2023-02-08 08:54:12,831 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4b057d76430e4231e80be3c284b60af5#family1#compaction#34 average throughput is 19.33 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:12,843 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/83959f5e67a249438b970d5b4997d07d, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/460c23ae7e8d4ff58999ab52750d2896, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/e6d595d0defa469ca661110ed146a94e, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/8d859ecf5685437794b88da43b39c1ea, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/539ae8a87c674194a38fc06820d31381, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/1bae165846fd4a9fae0d3c381bab8da4, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/27af5dbe6a0343cb852c58edb43211c4, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/14a88fba7e1a48a6a2779d25d9b0f27d, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/f3042eaf33654f808ed07af50167c7a3, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/21e48808703d4f2c9e4c8fc7b21166ee_SeqId_44_] to archive 2023-02-08 08:54:12,844 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:12,846 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/83959f5e67a249438b970d5b4997d07d to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/83959f5e67a249438b970d5b4997d07d 2023-02-08 08:54:12,848 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/460c23ae7e8d4ff58999ab52750d2896 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/460c23ae7e8d4ff58999ab52750d2896 2023-02-08 08:54:12,854 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/e6d595d0defa469ca661110ed146a94e to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/e6d595d0defa469ca661110ed146a94e 2023-02-08 08:54:12,856 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/8d859ecf5685437794b88da43b39c1ea to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/8d859ecf5685437794b88da43b39c1ea 2023-02-08 08:54:12,857 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/539ae8a87c674194a38fc06820d31381 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/539ae8a87c674194a38fc06820d31381 2023-02-08 08:54:12,859 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/1bae165846fd4a9fae0d3c381bab8da4 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/1bae165846fd4a9fae0d3c381bab8da4 2023-02-08 08:54:12,860 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/27af5dbe6a0343cb852c58edb43211c4 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/27af5dbe6a0343cb852c58edb43211c4 2023-02-08 08:54:12,862 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/14a88fba7e1a48a6a2779d25d9b0f27d to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/14a88fba7e1a48a6a2779d25d9b0f27d 2023-02-08 08:54:12,863 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/f3042eaf33654f808ed07af50167c7a3 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/f3042eaf33654f808ed07af50167c7a3 2023-02-08 08:54:12,864 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/21e48808703d4f2c9e4c8fc7b21166ee_SeqId_44_ to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/21e48808703d4f2c9e4c8fc7b21166ee_SeqId_44_ 2023-02-08 08:54:12,866 DEBUG [hfile_cleaner-dir-scan-pool-0] cleaner.CleanerChore(483): Couldn't delete 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy' yet because it isn't empty w/exception. org.apache.hadoop.fs.PathIsNotEmptyDirectoryException: ``/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) ': Directory is not empty: `/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1595) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:881) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:878) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:878) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$4(CleanerChore.java:453) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:478) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:453) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:455) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:61) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:439) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$2(CleanerChore.java:432) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): `/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.delete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:572) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1593) ... 29 more 2023-02-08 08:54:12,930 INFO [Listener at localhost/40763] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=253 (was 264), OpenFileDescriptor=601 (was 630), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=288 (was 231) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 168) - ProcessCount LEAK? -, AvailableMemoryMB=4469 (was 4725) 2023-02-08 08:54:12,941 INFO [Listener at localhost/40763] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=253, OpenFileDescriptor=601, MaxFileDescriptor=60000, SystemLoadAverage=288, ProcessCount=170, AvailableMemoryMB=4468 2023-02-08 08:54:12,941 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:12,941 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:12,943 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-08 08:54:12,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:12,946 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:54:12,946 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-02-08 08:54:12,946 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:12,946 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:12,947 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:54:12,948 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-08 08:54:12,949 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testScannerOnBulkLoadRefHFiles/70773be56fb60058a14d139eb59c7fb6 2023-02-08 08:54:12,949 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testScannerOnBulkLoadRefHFiles/0336b616da03fa0e018dfc24ffbd4919 2023-02-08 08:54:12,949 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6 2023-02-08 08:54:12,949 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testScannerOnBulkLoadRefHFiles/70773be56fb60058a14d139eb59c7fb6 empty. 2023-02-08 08:54:12,949 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testScannerOnBulkLoadRefHFiles/0336b616da03fa0e018dfc24ffbd4919 empty. 2023-02-08 08:54:12,949 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6 empty. 2023-02-08 08:54:12,950 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testScannerOnBulkLoadRefHFiles/70773be56fb60058a14d139eb59c7fb6 2023-02-08 08:54:12,950 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6 2023-02-08 08:54:12,950 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testScannerOnBulkLoadRefHFiles/0336b616da03fa0e018dfc24ffbd4919 2023-02-08 08:54:12,950 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-02-08 08:54:13,048 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:13,048 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:13,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-08 08:54:13,249 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:13,250 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:13,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-08 08:54:13,359 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-02-08 08:54:13,364 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7671): creating {ENCODED => 70773be56fb60058a14d139eb59c7fb6, NAME => 'testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp 2023-02-08 08:54:13,364 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7671): creating {ENCODED => 0336b616da03fa0e018dfc24ffbd4919, NAME => 'testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp 2023-02-08 08:54:13,368 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7671): creating {ENCODED => ddecc9a567f4f99742deec5e1f409ec6, NAME => 'testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp 2023-02-08 08:54:13,551 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:13,551 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:13,552 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-08 08:54:13,784 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:13,784 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1603): Closing 70773be56fb60058a14d139eb59c7fb6, disabling compactions & flushes 2023-02-08 08:54:13,784 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6. 2023-02-08 08:54:13,784 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6. 2023-02-08 08:54:13,785 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6. after waiting 0 ms 2023-02-08 08:54:13,785 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6. 2023-02-08 08:54:13,785 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6. 2023-02-08 08:54:13,785 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1557): Region close journal for 70773be56fb60058a14d139eb59c7fb6: Waiting for close lock at 1675846453784Disabling compacts and flushes for region at 1675846453784Disabling writes for close at 1675846453785 (+1 ms)Writing region close event to WAL at 1675846453785Closed at 1675846453785 2023-02-08 08:54:13,790 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:13,790 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1603): Closing 0336b616da03fa0e018dfc24ffbd4919, disabling compactions & flushes 2023-02-08 08:54:13,790 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919. 2023-02-08 08:54:13,790 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919. 2023-02-08 08:54:13,790 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919. after waiting 0 ms 2023-02-08 08:54:13,790 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919. 2023-02-08 08:54:13,790 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919. 2023-02-08 08:54:13,790 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1557): Region close journal for 0336b616da03fa0e018dfc24ffbd4919: Waiting for close lock at 1675846453790Disabling compacts and flushes for region at 1675846453790Disabling writes for close at 1675846453790Writing region close event to WAL at 1675846453790Closed at 1675846453790 2023-02-08 08:54:13,795 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:13,795 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1603): Closing ddecc9a567f4f99742deec5e1f409ec6, disabling compactions & flushes 2023-02-08 08:54:13,795 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6. 2023-02-08 08:54:13,795 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6. 2023-02-08 08:54:13,795 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6. after waiting 0 ms 2023-02-08 08:54:13,795 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6. 2023-02-08 08:54:13,795 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6. 2023-02-08 08:54:13,796 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1557): Region close journal for ddecc9a567f4f99742deec5e1f409ec6: Waiting for close lock at 1675846453795Disabling compacts and flushes for region at 1675846453795Disabling writes for close at 1675846453795Writing region close event to WAL at 1675846453795Closed at 1675846453795 2023-02-08 08:54:13,798 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:54:13,799 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675846453799"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846453799"}]},"ts":"1675846453799"} 2023-02-08 08:54:13,799 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675846453799"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846453799"}]},"ts":"1675846453799"} 2023-02-08 08:54:13,799 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675846453799"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846453799"}]},"ts":"1675846453799"} 2023-02-08 08:54:13,802 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-08 08:54:13,802 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:54:13,803 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846453802"}]},"ts":"1675846453802"} 2023-02-08 08:54:13,804 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-02-08 08:54:13,807 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=70773be56fb60058a14d139eb59c7fb6, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=0336b616da03fa0e018dfc24ffbd4919, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=ddecc9a567f4f99742deec5e1f409ec6, ASSIGN}] 2023-02-08 08:54:13,809 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=0336b616da03fa0e018dfc24ffbd4919, ASSIGN 2023-02-08 08:54:13,809 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=70773be56fb60058a14d139eb59c7fb6, ASSIGN 2023-02-08 08:54:13,809 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=ddecc9a567f4f99742deec5e1f409ec6, ASSIGN 2023-02-08 08:54:13,810 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=0336b616da03fa0e018dfc24ffbd4919, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=false 2023-02-08 08:54:13,810 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=ddecc9a567f4f99742deec5e1f409ec6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=false 2023-02-08 08:54:13,810 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=70773be56fb60058a14d139eb59c7fb6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=false 2023-02-08 08:54:13,961 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=70773be56fb60058a14d139eb59c7fb6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:13,961 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=0336b616da03fa0e018dfc24ffbd4919, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:13,961 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=ddecc9a567f4f99742deec5e1f409ec6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:13,962 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675846453961"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846453961"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846453961"}]},"ts":"1675846453961"} 2023-02-08 08:54:13,962 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675846453961"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846453961"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846453961"}]},"ts":"1675846453961"} 2023-02-08 08:54:13,962 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675846453961"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846453961"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846453961"}]},"ts":"1675846453961"} 2023-02-08 08:54:13,963 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE; OpenRegionProcedure 70773be56fb60058a14d139eb59c7fb6, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:54:13,964 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=30, state=RUNNABLE; OpenRegionProcedure ddecc9a567f4f99742deec5e1f409ec6, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:54:13,965 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=29, state=RUNNABLE; OpenRegionProcedure 0336b616da03fa0e018dfc24ffbd4919, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:54:14,053 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:14,053 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:14,054 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-08 08:54:14,066 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/.tmp/family1/3815f169e26b471bbc99418b1ebc57a3 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/3815f169e26b471bbc99418b1ebc57a3 2023-02-08 08:54:14,072 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 4b057d76430e4231e80be3c284b60af5/family1 of 4b057d76430e4231e80be3c284b60af5 into 3815f169e26b471bbc99418b1ebc57a3(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 08:54:14,073 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 4b057d76430e4231e80be3c284b60af5: 2023-02-08 08:54:14,073 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5., storeName=family1, fileCount=3, fileSize=176.8 K (165.1 K, 6.6 K, 5.2 K), priority=2, time=1675846452812; duration=1sec 2023-02-08 08:54:14,073 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:14,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919. 2023-02-08 08:54:14,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0336b616da03fa0e018dfc24ffbd4919, NAME => 'testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919.', STARTKEY => '1', ENDKEY => '2'} 2023-02-08 08:54:14,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 0336b616da03fa0e018dfc24ffbd4919 2023-02-08 08:54:14,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:14,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 0336b616da03fa0e018dfc24ffbd4919 2023-02-08 08:54:14,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 0336b616da03fa0e018dfc24ffbd4919 2023-02-08 08:54:14,121 INFO [StoreOpener-0336b616da03fa0e018dfc24ffbd4919-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 0336b616da03fa0e018dfc24ffbd4919 2023-02-08 08:54:14,122 DEBUG [StoreOpener-0336b616da03fa0e018dfc24ffbd4919-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/0336b616da03fa0e018dfc24ffbd4919/family1 2023-02-08 08:54:14,122 DEBUG [StoreOpener-0336b616da03fa0e018dfc24ffbd4919-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/0336b616da03fa0e018dfc24ffbd4919/family1 2023-02-08 08:54:14,123 INFO [StoreOpener-0336b616da03fa0e018dfc24ffbd4919-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0336b616da03fa0e018dfc24ffbd4919 columnFamilyName family1 2023-02-08 08:54:14,126 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/2636cf1ce76145ebb7d9109daa8a4df2, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/3577685d47204e728bc84194c2c51941, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/7dbcd903ab594ce68b3ade7eb3faeb52_SeqId_47_] to archive 2023-02-08 08:54:14,127 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:14,127 INFO [StoreOpener-0336b616da03fa0e018dfc24ffbd4919-1] regionserver.HStore(310): Store=0336b616da03fa0e018dfc24ffbd4919/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:14,128 INFO [StoreOpener-0336b616da03fa0e018dfc24ffbd4919-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 0336b616da03fa0e018dfc24ffbd4919 2023-02-08 08:54:14,129 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/2636cf1ce76145ebb7d9109daa8a4df2 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/2636cf1ce76145ebb7d9109daa8a4df2 2023-02-08 08:54:14,129 DEBUG [StoreOpener-0336b616da03fa0e018dfc24ffbd4919-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/0336b616da03fa0e018dfc24ffbd4919/family2 2023-02-08 08:54:14,129 DEBUG [StoreOpener-0336b616da03fa0e018dfc24ffbd4919-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/0336b616da03fa0e018dfc24ffbd4919/family2 2023-02-08 08:54:14,130 INFO [StoreOpener-0336b616da03fa0e018dfc24ffbd4919-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0336b616da03fa0e018dfc24ffbd4919 columnFamilyName family2 2023-02-08 08:54:14,130 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/3577685d47204e728bc84194c2c51941 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/3577685d47204e728bc84194c2c51941 2023-02-08 08:54:14,131 INFO [StoreOpener-0336b616da03fa0e018dfc24ffbd4919-1] regionserver.HStore(310): Store=0336b616da03fa0e018dfc24ffbd4919/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:14,132 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/7dbcd903ab594ce68b3ade7eb3faeb52_SeqId_47_ to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/family1/7dbcd903ab594ce68b3ade7eb3faeb52_SeqId_47_ 2023-02-08 08:54:14,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/0336b616da03fa0e018dfc24ffbd4919 2023-02-08 08:54:14,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/0336b616da03fa0e018dfc24ffbd4919 2023-02-08 08:54:14,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:54:14,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 0336b616da03fa0e018dfc24ffbd4919 2023-02-08 08:54:14,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/0336b616da03fa0e018dfc24ffbd4919/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:54:14,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 0336b616da03fa0e018dfc24ffbd4919; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72595107, jitterRate=0.08175139129161835}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:54:14,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 0336b616da03fa0e018dfc24ffbd4919: Running coprocessor pre-open hook at 1675846454119Writing region info on filesystem at 1675846454119Initializing all the Stores at 1675846454120 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846454120Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846454120Cleaning up temporary data from old regions at 1675846454134 (+14 ms)Cleaning up detritus from prior splits at 1675846454134Running coprocessor post-open hooks at 1675846454139 (+5 ms)Region opened successfully at 1675846454139 2023-02-08 08:54:14,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919., pid=33, masterSystemTime=1675846454115 2023-02-08 08:54:14,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919. 2023-02-08 08:54:14,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919. 2023-02-08 08:54:14,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6. 2023-02-08 08:54:14,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ddecc9a567f4f99742deec5e1f409ec6, NAME => 'testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6.', STARTKEY => '2', ENDKEY => ''} 2023-02-08 08:54:14,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles ddecc9a567f4f99742deec5e1f409ec6 2023-02-08 08:54:14,142 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=0336b616da03fa0e018dfc24ffbd4919, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:14,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:14,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for ddecc9a567f4f99742deec5e1f409ec6 2023-02-08 08:54:14,142 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1675846454142"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846454142"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846454142"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846454142"}]},"ts":"1675846454142"} 2023-02-08 08:54:14,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for ddecc9a567f4f99742deec5e1f409ec6 2023-02-08 08:54:14,143 INFO [StoreOpener-ddecc9a567f4f99742deec5e1f409ec6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region ddecc9a567f4f99742deec5e1f409ec6 2023-02-08 08:54:14,145 DEBUG [StoreOpener-ddecc9a567f4f99742deec5e1f409ec6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1 2023-02-08 08:54:14,145 DEBUG [StoreOpener-ddecc9a567f4f99742deec5e1f409ec6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1 2023-02-08 08:54:14,146 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=29 2023-02-08 08:54:14,146 INFO [StoreOpener-ddecc9a567f4f99742deec5e1f409ec6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ddecc9a567f4f99742deec5e1f409ec6 columnFamilyName family1 2023-02-08 08:54:14,146 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=29, state=SUCCESS; OpenRegionProcedure 0336b616da03fa0e018dfc24ffbd4919, server=jenkins-hbase4.apache.org,42239,1675846413348 in 179 msec 2023-02-08 08:54:14,147 INFO [StoreOpener-ddecc9a567f4f99742deec5e1f409ec6-1] regionserver.HStore(310): Store=ddecc9a567f4f99742deec5e1f409ec6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:14,147 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=0336b616da03fa0e018dfc24ffbd4919, ASSIGN in 339 msec 2023-02-08 08:54:14,147 INFO [StoreOpener-ddecc9a567f4f99742deec5e1f409ec6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region ddecc9a567f4f99742deec5e1f409ec6 2023-02-08 08:54:14,149 DEBUG [StoreOpener-ddecc9a567f4f99742deec5e1f409ec6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family2 2023-02-08 08:54:14,149 DEBUG [StoreOpener-ddecc9a567f4f99742deec5e1f409ec6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family2 2023-02-08 08:54:14,150 INFO [StoreOpener-ddecc9a567f4f99742deec5e1f409ec6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ddecc9a567f4f99742deec5e1f409ec6 columnFamilyName family2 2023-02-08 08:54:14,151 INFO [StoreOpener-ddecc9a567f4f99742deec5e1f409ec6-1] regionserver.HStore(310): Store=ddecc9a567f4f99742deec5e1f409ec6/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:14,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6 2023-02-08 08:54:14,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6 2023-02-08 08:54:14,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:54:14,155 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for ddecc9a567f4f99742deec5e1f409ec6 2023-02-08 08:54:14,157 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:54:14,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened ddecc9a567f4f99742deec5e1f409ec6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68080520, jitterRate=0.014478802680969238}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:54:14,157 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for ddecc9a567f4f99742deec5e1f409ec6: Running coprocessor pre-open hook at 1675846454142Writing region info on filesystem at 1675846454142Initializing all the Stores at 1675846454143 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846454143Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846454143Cleaning up temporary data from old regions at 1675846454153 (+10 ms)Cleaning up detritus from prior splits at 1675846454153Running coprocessor post-open hooks at 1675846454157 (+4 ms)Region opened successfully at 1675846454157 2023-02-08 08:54:14,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6., pid=32, masterSystemTime=1675846454115 2023-02-08 08:54:14,159 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6. 2023-02-08 08:54:14,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6. 2023-02-08 08:54:14,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6. 2023-02-08 08:54:14,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 70773be56fb60058a14d139eb59c7fb6, NAME => 'testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6.', STARTKEY => '', ENDKEY => '1'} 2023-02-08 08:54:14,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 70773be56fb60058a14d139eb59c7fb6 2023-02-08 08:54:14,160 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=ddecc9a567f4f99742deec5e1f409ec6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:14,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:14,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 70773be56fb60058a14d139eb59c7fb6 2023-02-08 08:54:14,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 70773be56fb60058a14d139eb59c7fb6 2023-02-08 08:54:14,160 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675846454160"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846454160"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846454160"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846454160"}]},"ts":"1675846454160"} 2023-02-08 08:54:14,161 INFO [StoreOpener-70773be56fb60058a14d139eb59c7fb6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 70773be56fb60058a14d139eb59c7fb6 2023-02-08 08:54:14,162 DEBUG [StoreOpener-70773be56fb60058a14d139eb59c7fb6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/70773be56fb60058a14d139eb59c7fb6/family1 2023-02-08 08:54:14,163 DEBUG [StoreOpener-70773be56fb60058a14d139eb59c7fb6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/70773be56fb60058a14d139eb59c7fb6/family1 2023-02-08 08:54:14,163 INFO [StoreOpener-70773be56fb60058a14d139eb59c7fb6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 70773be56fb60058a14d139eb59c7fb6 columnFamilyName family1 2023-02-08 08:54:14,163 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=30 2023-02-08 08:54:14,163 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=30, state=SUCCESS; OpenRegionProcedure ddecc9a567f4f99742deec5e1f409ec6, server=jenkins-hbase4.apache.org,42239,1675846413348 in 197 msec 2023-02-08 08:54:14,164 INFO [StoreOpener-70773be56fb60058a14d139eb59c7fb6-1] regionserver.HStore(310): Store=70773be56fb60058a14d139eb59c7fb6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:14,164 INFO [StoreOpener-70773be56fb60058a14d139eb59c7fb6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 70773be56fb60058a14d139eb59c7fb6 2023-02-08 08:54:14,165 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=ddecc9a567f4f99742deec5e1f409ec6, ASSIGN in 356 msec 2023-02-08 08:54:14,166 DEBUG [StoreOpener-70773be56fb60058a14d139eb59c7fb6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/70773be56fb60058a14d139eb59c7fb6/family2 2023-02-08 08:54:14,166 DEBUG [StoreOpener-70773be56fb60058a14d139eb59c7fb6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/70773be56fb60058a14d139eb59c7fb6/family2 2023-02-08 08:54:14,166 INFO [StoreOpener-70773be56fb60058a14d139eb59c7fb6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 70773be56fb60058a14d139eb59c7fb6 columnFamilyName family2 2023-02-08 08:54:14,167 INFO [StoreOpener-70773be56fb60058a14d139eb59c7fb6-1] regionserver.HStore(310): Store=70773be56fb60058a14d139eb59c7fb6/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:14,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/70773be56fb60058a14d139eb59c7fb6 2023-02-08 08:54:14,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/70773be56fb60058a14d139eb59c7fb6 2023-02-08 08:54:14,170 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:54:14,171 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 70773be56fb60058a14d139eb59c7fb6 2023-02-08 08:54:14,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/70773be56fb60058a14d139eb59c7fb6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:54:14,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 70773be56fb60058a14d139eb59c7fb6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71974536, jitterRate=0.07250416278839111}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:54:14,173 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 70773be56fb60058a14d139eb59c7fb6: Running coprocessor pre-open hook at 1675846454160Writing region info on filesystem at 1675846454160Initializing all the Stores at 1675846454161 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846454161Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846454161Cleaning up temporary data from old regions at 1675846454169 (+8 ms)Cleaning up detritus from prior splits at 1675846454169Running coprocessor post-open hooks at 1675846454173 (+4 ms)Region opened successfully at 1675846454173 2023-02-08 08:54:14,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6., pid=31, masterSystemTime=1675846454115 2023-02-08 08:54:14,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6. 2023-02-08 08:54:14,176 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6. 2023-02-08 08:54:14,176 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=70773be56fb60058a14d139eb59c7fb6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:14,177 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1675846454176"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846454176"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846454176"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846454176"}]},"ts":"1675846454176"} 2023-02-08 08:54:14,179 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-02-08 08:54:14,179 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; OpenRegionProcedure 70773be56fb60058a14d139eb59c7fb6, server=jenkins-hbase4.apache.org,42239,1675846413348 in 215 msec 2023-02-08 08:54:14,181 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=28, resume processing ppid=27 2023-02-08 08:54:14,181 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=70773be56fb60058a14d139eb59c7fb6, ASSIGN in 372 msec 2023-02-08 08:54:14,182 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:54:14,182 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846454182"}]},"ts":"1675846454182"} 2023-02-08 08:54:14,183 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-02-08 08:54:14,185 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:54:14,186 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 1.2420 sec 2023-02-08 08:54:15,054 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:15,054 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:15,055 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-08 08:54:15,055 INFO [Listener at localhost/40763] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-02-08 08:54:15,062 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:15,062 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:15,062 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:15,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,064 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-02-08 08:54:15,064 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:15,064 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-02-08 08:54:15,064 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-02-08 08:54:15,065 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,065 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,066 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:15,066 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:15,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:15,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:15,066 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:15,066 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:15,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,067 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:15,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,067 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,067 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,067 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-02-08 08:54:15,067 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:15,067 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-02-08 08:54:15,069 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6. 2023-02-08 08:54:15,070 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6. started... 2023-02-08 08:54:15,070 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919. 2023-02-08 08:54:15,070 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919. started... 2023-02-08 08:54:15,072 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 08:54:15,072 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:15,072 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 70773be56fb60058a14d139eb59c7fb6: 2023-02-08 08:54:15,072 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6. 2023-02-08 08:54:15,073 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6. started... 2023-02-08 08:54:15,072 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 0336b616da03fa0e018dfc24ffbd4919: 2023-02-08 08:54:15,073 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919. 2023-02-08 08:54:15,073 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing ddecc9a567f4f99742deec5e1f409ec6 2/2 column families, dataSize=48 B heapSize=632 B 2023-02-08 08:54:15,073 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 08:54:15,072 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6. 2023-02-08 08:54:15,073 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 08:54:15,486 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208647e75af64a44058b5e1c9895dad68f8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230208647e75af64a44058b5e1c9895dad68f8 2023-02-08 08:54:15,892 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/.tmp/family1/cef6e0da71c342b69e6f217d4cb64654 2023-02-08 08:54:15,897 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/.tmp/family1/cef6e0da71c342b69e6f217d4cb64654 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1/cef6e0da71c342b69e6f217d4cb64654 2023-02-08 08:54:15,901 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1/cef6e0da71c342b69e6f217d4cb64654, entries=1, sequenceid=5, filesize=4.9 K 2023-02-08 08:54:15,902 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for ddecc9a567f4f99742deec5e1f409ec6 in 829ms, sequenceid=5, compaction requested=false 2023-02-08 08:54:15,902 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-02-08 08:54:15,902 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for ddecc9a567f4f99742deec5e1f409ec6: 2023-02-08 08:54:15,902 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6. 2023-02-08 08:54:15,902 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 08:54:15,902 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 08:54:15,902 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:15,902 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-02-08 08:54:15,903 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-02-08 08:54:15,904 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:15,904 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:15,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:15,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:15,905 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,905 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:15,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:15,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:15,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:15,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:15,906 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-02-08 08:54:15,906 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7f4dbe04[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:15,906 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-02-08 08:54:15,906 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,907 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,908 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-02-08 08:54:15,908 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:15,908 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-02-08 08:54:15,908 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:15,908 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:15,910 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:15,910 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-02-08 08:54:15,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:15,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:15,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:15,910 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:15,910 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-02-08 08:54:15,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:15,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:15,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:15,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:15,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:15,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:15,912 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-02-08 08:54:15,912 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-02-08 08:54:15,912 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:15,912 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:15,912 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,912 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:15,914 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,914 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:15,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:15,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,914 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:15,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,914 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:15,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:15,914 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:15,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:15,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:15,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,915 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:15,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:15,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:15,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,921 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:15,921 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:15,921 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:15,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:15,921 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-02-08 08:54:15,921 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:15,921 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:15,921 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,921 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-02-08 08:54:15,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:15,921 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,921 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:15,921 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:15,921 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,921 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:15,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:15,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:15,921 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:15,922 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,022 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:16,022 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:16,022 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:16,023 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:16,026 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:16,026 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:16,027 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:16,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:16,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,029 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-02-08 08:54:16,029 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,029 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-02-08 08:54:16,029 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-02-08 08:54:16,030 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,030 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,031 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,031 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:16,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,032 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:16,032 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:16,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,032 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:16,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,033 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-02-08 08:54:16,033 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,033 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-02-08 08:54:16,033 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:16,033 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-02-08 08:54:16,033 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6. 2023-02-08 08:54:16,033 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919. 2023-02-08 08:54:16,033 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 08:54:16,033 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919. started... 2023-02-08 08:54:16,033 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6. started... 2023-02-08 08:54:16,034 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:16,034 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 0336b616da03fa0e018dfc24ffbd4919: 2023-02-08 08:54:16,034 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 70773be56fb60058a14d139eb59c7fb6: 2023-02-08 08:54:16,034 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6. 2023-02-08 08:54:16,034 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6. 2023-02-08 08:54:16,034 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919. 2023-02-08 08:54:16,034 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 08:54:16,034 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6. started... 2023-02-08 08:54:16,034 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 08:54:16,034 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing ddecc9a567f4f99742deec5e1f409ec6 2/2 column families, dataSize=90 B heapSize=752 B 2023-02-08 08:54:16,450 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302087792a2fa691745f39df0192e2bbb41b2 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202302087792a2fa691745f39df0192e2bbb41b2 2023-02-08 08:54:16,856 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/.tmp/family1/a6d2e813a2294a6eabca08888366ea08 2023-02-08 08:54:16,862 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/.tmp/family1/a6d2e813a2294a6eabca08888366ea08 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1/a6d2e813a2294a6eabca08888366ea08 2023-02-08 08:54:16,866 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1/a6d2e813a2294a6eabca08888366ea08, entries=2, sequenceid=9, filesize=5.0 K 2023-02-08 08:54:16,867 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for ddecc9a567f4f99742deec5e1f409ec6 in 833ms, sequenceid=9, compaction requested=false 2023-02-08 08:54:16,868 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for ddecc9a567f4f99742deec5e1f409ec6: 2023-02-08 08:54:16,868 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6. 2023-02-08 08:54:16,868 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 08:54:16,868 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 08:54:16,868 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:16,868 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-02-08 08:54:16,868 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-02-08 08:54:16,870 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:16,870 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:16,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:16,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:16,870 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,870 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:16,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:16,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:16,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:16,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:16,872 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-02-08 08:54:16,872 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b96a1d5[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:16,872 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-02-08 08:54:16,872 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,873 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,873 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,874 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-02-08 08:54:16,874 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-02-08 08:54:16,874 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:16,874 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:16,874 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:16,875 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-02-08 08:54:16,875 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:16,875 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:16,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:16,875 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-02-08 08:54:16,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:16,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:16,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:16,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:16,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:16,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:16,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:16,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:16,878 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-02-08 08:54:16,878 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-02-08 08:54:16,878 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:16,878 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:16,878 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,878 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:16,881 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,881 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,881 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:16,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:16,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:16,882 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:16,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:16,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:16,883 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:16,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:16,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:16,885 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:16,885 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:16,885 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,885 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:16,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:16,885 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-02-08 08:54:16,885 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:16,885 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:16,885 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:16,885 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-02-08 08:54:16,885 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,885 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:16,885 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,886 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,886 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,886 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:16,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:16,886 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:16,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:16,986 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:16,986 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:16,986 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:16,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:16,995 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37869] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-02-08 08:54:16,996 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-08 08:54:16,997 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-08 08:54:16,998 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:54:16,998 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675847056998 2023-02-08 08:54:16,998 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 08:54:16,999 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-02-08 08:54:16,999 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-08 08:54:16,999 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] zookeeper.ReadOnlyZKClient(139): Connect 0x7706c0e8 to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:54:17,003 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@414b44f2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:54:17,004 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230208 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-02-08 08:54:17,833 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208fc21e5f1e80a42a198d03f9121778c6c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230208fc21e5f1e80a42a198d03f9121778c6c 2023-02-08 08:54:17,833 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230208 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-08 08:54:17,847 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:54:17,848 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:32912, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:54:17,855 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:17,855 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:17,856 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 08:54:17,858 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40780, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 08:54:17,867 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230208/family1/ec14e1551f5140ff9692ef4907af2652 first=Optional[k0] last=Optional[k1] 2023-02-08 08:54:17,868 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230208/family1/ec14e1551f5140ff9692ef4907af2652}] 2023-02-08 08:54:17,872 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230208/family1/ec14e1551f5140ff9692ef4907af2652 for inclusion in ddecc9a567f4f99742deec5e1f409ec6/family1 2023-02-08 08:54:17,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-02-08 08:54:17,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-08 08:54:17,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for ddecc9a567f4f99742deec5e1f409ec6: 2023-02-08 08:54:17,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230208/family1/ec14e1551f5140ff9692ef4907af2652 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testScannerOnBulkLoadRefHFiles__8egultij79p40n0ltr7s02t0ketigodqcbrrs6ffjlhdntau6l73j32ta0k2mj0f/family1/ec14e1551f5140ff9692ef4907af2652 2023-02-08 08:54:17,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testScannerOnBulkLoadRefHFiles__8egultij79p40n0ltr7s02t0ketigodqcbrrs6ffjlhdntau6l73j32ta0k2mj0f/family1/ec14e1551f5140ff9692ef4907af2652 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1/70d132b5a781489890d499d3812b5a5e_SeqId_12_ 2023-02-08 08:54:17,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testScannerOnBulkLoadRefHFiles__8egultij79p40n0ltr7s02t0ketigodqcbrrs6ffjlhdntau6l73j32ta0k2mj0f/family1/ec14e1551f5140ff9692ef4907af2652 into ddecc9a567f4f99742deec5e1f409ec6/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1/70d132b5a781489890d499d3812b5a5e_SeqId_12_ - updating store file list. 2023-02-08 08:54:17,882 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1/70d132b5a781489890d499d3812b5a5e_SeqId_12_ into ddecc9a567f4f99742deec5e1f409ec6/family1 2023-02-08 08:54:17,882 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testScannerOnBulkLoadRefHFiles__8egultij79p40n0ltr7s02t0ketigodqcbrrs6ffjlhdntau6l73j32ta0k2mj0f/family1/ec14e1551f5140ff9692ef4907af2652 into ddecc9a567f4f99742deec5e1f409ec6/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1/70d132b5a781489890d499d3812b5a5e_SeqId_12_) 2023-02-08 08:54:17,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testScannerOnBulkLoadRefHFiles__8egultij79p40n0ltr7s02t0ketigodqcbrrs6ffjlhdntau6l73j32ta0k2mj0f/family1/ec14e1551f5140ff9692ef4907af2652 2023-02-08 08:54:17,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-08 08:54:17,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-08 08:54:17,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(1912): ddecc9a567f4f99742deec5e1f409ec6/family1 is initiating minor compaction (all files) 2023-02-08 08:54:17,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@42308d6; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:17,884 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of ddecc9a567f4f99742deec5e1f409ec6/family1 in testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6. 2023-02-08 08:54:17,884 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1/cef6e0da71c342b69e6f217d4cb64654, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1/a6d2e813a2294a6eabca08888366ea08, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1/70d132b5a781489890d499d3812b5a5e_SeqId_12_] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/.tmp, totalSize=15.1 K 2023-02-08 08:54:17,885 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting cef6e0da71c342b69e6f217d4cb64654, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846455055 2023-02-08 08:54:17,885 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting a6d2e813a2294a6eabca08888366ea08, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846455055 2023-02-08 08:54:17,885 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 70d132b5a781489890d499d3812b5a5e_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846455055 2023-02-08 08:54:17,886 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230208 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-08 08:54:17,887 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:17,889 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230208647e75af64a44058b5e1c9895dad68f8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230208647e75af64a44058b5e1c9895dad68f8 2023-02-08 08:54:17,893 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202302087792a2fa691745f39df0192e2bbb41b2 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202302087792a2fa691745f39df0192e2bbb41b2 2023-02-08 08:54:17,893 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-02-08 08:54:17,893 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230208' 2023-02-08 08:54:17,893 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230208 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-02-08 08:54:17,893 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-08 08:54:17,893 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:54:17,894 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-08 08:54:17,896 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 899 msec 2023-02-08 08:54:17,897 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] throttle.PressureAwareThroughputController(145): ddecc9a567f4f99742deec5e1f409ec6#family1#compaction#37 average throughput is 0.05 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:17,930 INFO [Listener at localhost/40763] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=251 (was 253), OpenFileDescriptor=600 (was 601), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=321 (was 288) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 170), AvailableMemoryMB=4334 (was 4468) 2023-02-08 08:54:17,940 INFO [Listener at localhost/40763] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=251, OpenFileDescriptor=600, MaxFileDescriptor=60000, SystemLoadAverage=321, ProcessCount=168, AvailableMemoryMB=4333 2023-02-08 08:54:17,941 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:17,941 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:17,942 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-08 08:54:17,943 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:17,944 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:54:17,944 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-02-08 08:54:17,945 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:17,945 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:17,945 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:54:17,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-08 08:54:17,946 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd 2023-02-08 08:54:17,947 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd empty. 2023-02-08 08:54:17,947 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd 2023-02-08 08:54:17,947 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-02-08 08:54:18,045 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:18,046 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:18,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-08 08:54:18,247 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:18,247 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:18,248 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-08 08:54:18,355 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-08 08:54:18,356 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => b7daddc3345a00410a8096f6715baffd, NAME => 'testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp 2023-02-08 08:54:18,548 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:18,548 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:18,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-08 08:54:18,763 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:18,763 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1603): Closing b7daddc3345a00410a8096f6715baffd, disabling compactions & flushes 2023-02-08 08:54:18,763 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:18,763 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:18,763 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. after waiting 0 ms 2023-02-08 08:54:18,763 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:18,763 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:18,763 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for b7daddc3345a00410a8096f6715baffd: Waiting for close lock at 1675846458763Disabling compacts and flushes for region at 1675846458763Disabling writes for close at 1675846458763Writing region close event to WAL at 1675846458763Closed at 1675846458763 2023-02-08 08:54:18,765 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:54:18,766 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675846458766"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846458766"}]},"ts":"1675846458766"} 2023-02-08 08:54:18,768 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 08:54:18,769 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:54:18,769 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846458769"}]},"ts":"1675846458769"} 2023-02-08 08:54:18,770 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-02-08 08:54:18,774 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=b7daddc3345a00410a8096f6715baffd, ASSIGN}] 2023-02-08 08:54:18,775 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=b7daddc3345a00410a8096f6715baffd, ASSIGN 2023-02-08 08:54:18,776 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=b7daddc3345a00410a8096f6715baffd, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=false 2023-02-08 08:54:18,927 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=b7daddc3345a00410a8096f6715baffd, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:18,927 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675846458927"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846458927"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846458927"}]},"ts":"1675846458927"} 2023-02-08 08:54:18,929 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure b7daddc3345a00410a8096f6715baffd, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:54:19,049 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:19,049 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:19,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-08 08:54:19,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:19,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b7daddc3345a00410a8096f6715baffd, NAME => 'testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:54:19,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy b7daddc3345a00410a8096f6715baffd 2023-02-08 08:54:19,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:19,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b7daddc3345a00410a8096f6715baffd 2023-02-08 08:54:19,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b7daddc3345a00410a8096f6715baffd 2023-02-08 08:54:19,085 INFO [StoreOpener-b7daddc3345a00410a8096f6715baffd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b7daddc3345a00410a8096f6715baffd 2023-02-08 08:54:19,086 DEBUG [StoreOpener-b7daddc3345a00410a8096f6715baffd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1 2023-02-08 08:54:19,086 DEBUG [StoreOpener-b7daddc3345a00410a8096f6715baffd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1 2023-02-08 08:54:19,087 INFO [StoreOpener-b7daddc3345a00410a8096f6715baffd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b7daddc3345a00410a8096f6715baffd columnFamilyName family1 2023-02-08 08:54:19,088 INFO [StoreOpener-b7daddc3345a00410a8096f6715baffd-1] regionserver.HStore(310): Store=b7daddc3345a00410a8096f6715baffd/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:19,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd 2023-02-08 08:54:19,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd 2023-02-08 08:54:19,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b7daddc3345a00410a8096f6715baffd 2023-02-08 08:54:19,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:54:19,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b7daddc3345a00410a8096f6715baffd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60543319, jitterRate=-0.09783424437046051}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 08:54:19,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b7daddc3345a00410a8096f6715baffd: Running coprocessor pre-open hook at 1675846459084Writing region info on filesystem at 1675846459084Initializing all the Stores at 1675846459085 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675846459085Cleaning up temporary data from old regions at 1675846459090 (+5 ms)Cleaning up detritus from prior splits at 1675846459090Running coprocessor post-open hooks at 1675846459094 (+4 ms)Region opened successfully at 1675846459094 2023-02-08 08:54:19,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd., pid=37, masterSystemTime=1675846459081 2023-02-08 08:54:19,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:19,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:19,097 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=b7daddc3345a00410a8096f6715baffd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:19,097 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1675846459097"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846459097"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846459097"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846459097"}]},"ts":"1675846459097"} 2023-02-08 08:54:19,100 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-02-08 08:54:19,100 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure b7daddc3345a00410a8096f6715baffd, server=jenkins-hbase4.apache.org,42239,1675846413348 in 169 msec 2023-02-08 08:54:19,101 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-08 08:54:19,101 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=b7daddc3345a00410a8096f6715baffd, ASSIGN in 326 msec 2023-02-08 08:54:19,102 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:54:19,102 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846459102"}]},"ts":"1675846459102"} 2023-02-08 08:54:19,103 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-02-08 08:54:19,105 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:54:19,106 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 1.1630 sec 2023-02-08 08:54:19,116 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/.tmp/family1/036c52df84ee4d1d8e0ce11484f585cc as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1/036c52df84ee4d1d8e0ce11484f585cc 2023-02-08 08:54:19,120 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in ddecc9a567f4f99742deec5e1f409ec6/family1 of ddecc9a567f4f99742deec5e1f409ec6 into 036c52df84ee4d1d8e0ce11484f585cc(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 08:54:19,120 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for ddecc9a567f4f99742deec5e1f409ec6: 2023-02-08 08:54:19,120 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1675846457884; duration=1sec 2023-02-08 08:54:19,120 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:19,134 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1/cef6e0da71c342b69e6f217d4cb64654, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1/a6d2e813a2294a6eabca08888366ea08, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1/70d132b5a781489890d499d3812b5a5e_SeqId_12_] to archive 2023-02-08 08:54:19,135 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:19,137 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1/cef6e0da71c342b69e6f217d4cb64654 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1/cef6e0da71c342b69e6f217d4cb64654 2023-02-08 08:54:19,138 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1/a6d2e813a2294a6eabca08888366ea08 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1/a6d2e813a2294a6eabca08888366ea08 2023-02-08 08:54:19,139 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1/70d132b5a781489890d499d3812b5a5e_SeqId_12_ to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/family1/70d132b5a781489890d499d3812b5a5e_SeqId_12_ 2023-02-08 08:54:19,648 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 08:54:20,050 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:20,051 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:20,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-08 08:54:20,052 INFO [Listener at localhost/40763] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-02-08 08:54:20,080 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:20,081 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:20,081 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:20,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,083 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 08:54:20,083 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:20,083 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 08:54:20,083 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:20,084 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,084 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,085 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:20,085 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:20,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:20,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:20,085 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:20,085 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:20,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,086 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:20,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,086 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,086 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,086 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 08:54:20,087 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:20,087 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 08:54:20,087 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:20,087 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:20,087 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:20,087 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. started... 2023-02-08 08:54:20,088 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b7daddc3345a00410a8096f6715baffd 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-08 08:54:20,511 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511306080048901c24e1097688267690bdd71 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511306080048901c24e1097688267690bdd71 2023-02-08 08:54:20,915 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/a4061aaf1fb44546a4d35af015c732ed 2023-02-08 08:54:20,920 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/a4061aaf1fb44546a4d35af015c732ed as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/a4061aaf1fb44546a4d35af015c732ed 2023-02-08 08:54:20,924 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/a4061aaf1fb44546a4d35af015c732ed, entries=1000, sequenceid=5, filesize=165.1 K 2023-02-08 08:54:20,925 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for b7daddc3345a00410a8096f6715baffd in 838ms, sequenceid=5, compaction requested=false 2023-02-08 08:54:20,925 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 08:54:20,925 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b7daddc3345a00410a8096f6715baffd: 2023-02-08 08:54:20,925 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:20,926 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:20,926 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:20,926 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:20,926 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 08:54:20,926 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-08 08:54:20,927 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:20,927 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:20,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:20,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:20,927 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,927 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:20,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:20,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:20,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:20,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:20,929 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 08:54:20,929 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72451470[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:20,929 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:20,929 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,930 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,930 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:20,930 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-08 08:54:20,930 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:20,930 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:20,930 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:20,932 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:20,932 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 08:54:20,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:20,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:20,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:20,932 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:20,932 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-08 08:54:20,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:20,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:20,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:20,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:20,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:20,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:20,934 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:20,934 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-08 08:54:20,934 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:20,934 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:20,934 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,934 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:20,936 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,936 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:20,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:20,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,936 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:20,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:20,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:20,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:20,936 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:20,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:20,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,937 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:20,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:20,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:20,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:20,939 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:20,939 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,939 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:20,939 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:20,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 08:54:20,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:20,939 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:20,939 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 08:54:20,940 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:20,939 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:20,940 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:20,940 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,940 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,940 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:20,940 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:20,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:20,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:20,940 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:21,040 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:21,040 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:21,040 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:21,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:21,045 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:21,045 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:21,046 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:21,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:21,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,047 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 08:54:21,048 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:21,048 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 08:54:21,048 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:21,048 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,048 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,049 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:21,049 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:21,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:21,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:21,049 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:21,049 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:21,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,050 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:21,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,050 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,050 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 08:54:21,050 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,050 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 08:54:21,050 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:21,051 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 08:54:21,051 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:21,051 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:21,051 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:21,051 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. started... 2023-02-08 08:54:21,051 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b7daddc3345a00410a8096f6715baffd 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-08 08:54:21,466 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128dc30bad2233b4e94a82a9f302e5a7e51 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151128dc30bad2233b4e94a82a9f302e5a7e51 2023-02-08 08:54:21,871 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/506f31968af04443b20c031965715602 2023-02-08 08:54:21,876 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/506f31968af04443b20c031965715602 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/506f31968af04443b20c031965715602 2023-02-08 08:54:21,879 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/506f31968af04443b20c031965715602, entries=1, sequenceid=9, filesize=5.0 K 2023-02-08 08:54:21,880 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for b7daddc3345a00410a8096f6715baffd in 829ms, sequenceid=9, compaction requested=false 2023-02-08 08:54:21,880 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b7daddc3345a00410a8096f6715baffd: 2023-02-08 08:54:21,880 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:21,880 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:21,880 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:21,880 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:21,880 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 08:54:21,881 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-08 08:54:21,883 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:21,883 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:21,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:21,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:21,883 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,883 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:21,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:21,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:21,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:21,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:21,884 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 08:54:21,884 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6484ef8[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:21,884 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:21,885 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,888 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,888 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:21,888 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-08 08:54:21,888 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:21,888 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:21,889 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:21,890 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 08:54:21,890 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:21,890 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:21,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:21,890 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-08 08:54:21,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:21,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:21,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:21,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:21,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:21,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:21,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:21,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:21,892 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:21,892 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-08 08:54:21,892 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:21,892 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:21,892 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,892 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:21,895 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,895 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,895 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:21,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:21,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:21,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,895 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:21,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:21,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:21,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:21,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:21,896 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:21,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:21,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:21,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:21,898 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:21,898 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:21,898 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,898 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:21,898 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:21,898 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 08:54:21,898 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:21,898 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,898 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:21,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:21,898 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 08:54:21,898 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:21,898 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,898 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:21,899 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:21,899 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:21,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:21,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:21,999 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:21,999 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:21,999 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:22,000 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:22,003 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:22,003 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:22,004 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:22,005 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:22,005 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,005 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 08:54:22,005 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:22,005 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 08:54:22,005 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:22,006 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,006 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,007 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:22,007 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:22,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:22,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:22,007 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:22,007 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:22,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,008 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:22,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,008 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,008 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 08:54:22,008 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,008 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 08:54:22,008 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:22,008 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 08:54:22,009 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:22,009 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:22,009 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:22,009 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. started... 2023-02-08 08:54:22,009 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b7daddc3345a00410a8096f6715baffd 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:22,421 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201da734f8416464deb9104fb351eee7edb to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201da734f8416464deb9104fb351eee7edb 2023-02-08 08:54:22,827 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/3edd5addbded4b4a9fbef04b8f4d1c30 2023-02-08 08:54:22,832 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/3edd5addbded4b4a9fbef04b8f4d1c30 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/3edd5addbded4b4a9fbef04b8f4d1c30 2023-02-08 08:54:22,836 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/3edd5addbded4b4a9fbef04b8f4d1c30, entries=1, sequenceid=13, filesize=4.9 K 2023-02-08 08:54:22,837 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b7daddc3345a00410a8096f6715baffd in 828ms, sequenceid=13, compaction requested=true 2023-02-08 08:54:22,837 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b7daddc3345a00410a8096f6715baffd: 2023-02-08 08:54:22,837 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:22,837 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:22,837 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:22,837 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:22,838 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 08:54:22,838 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-08 08:54:22,839 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:22,839 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:22,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:22,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:22,840 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:22,840 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:22,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:22,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:22,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:22,841 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 08:54:22,841 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@14e77dab[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:22,841 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:22,841 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,843 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,843 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:22,843 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:22,843 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-08 08:54:22,843 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:22,843 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:22,844 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 08:54:22,844 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:22,844 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:22,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:22,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:22,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:22,845 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-08 08:54:22,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:22,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:22,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:22,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:22,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:22,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:22,847 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:22,847 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-08 08:54:22,847 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:22,847 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:22,847 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,847 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:22,848 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,848 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,848 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:22,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:22,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,848 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:22,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:22,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:22,849 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:22,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:22,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:22,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,850 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:22,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:22,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:22,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:22,852 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:22,852 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:22,852 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:22,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:22,852 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:22,852 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 08:54:22,852 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:22,852 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:22,852 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,852 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:22,852 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,852 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,852 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,852 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:22,852 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:22,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:22,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:22,953 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:22,953 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:22,953 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:22,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:22,957 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:22,957 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:22,957 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:22,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:22,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,959 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 08:54:22,959 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:22,959 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 08:54:22,959 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:22,960 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,960 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,962 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:22,962 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:22,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:22,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:22,962 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:22,962 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:22,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,962 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:22,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,963 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,963 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 08:54:22,963 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:22,963 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 08:54:22,963 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:22,963 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 08:54:22,963 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:22,963 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:22,963 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:22,963 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. started... 2023-02-08 08:54:22,964 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b7daddc3345a00410a8096f6715baffd 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:23,377 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205c756982438de4f97bebd3aa2a2a0cf2e to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205c756982438de4f97bebd3aa2a2a0cf2e 2023-02-08 08:54:23,783 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/3948e90b496647b0999bc0d714ecd381 2023-02-08 08:54:23,787 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/3948e90b496647b0999bc0d714ecd381 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/3948e90b496647b0999bc0d714ecd381 2023-02-08 08:54:23,791 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/3948e90b496647b0999bc0d714ecd381, entries=1, sequenceid=17, filesize=4.9 K 2023-02-08 08:54:23,792 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b7daddc3345a00410a8096f6715baffd in 828ms, sequenceid=17, compaction requested=true 2023-02-08 08:54:23,793 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b7daddc3345a00410a8096f6715baffd: 2023-02-08 08:54:23,793 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:23,793 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:23,793 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:23,793 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:23,793 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 08:54:23,793 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-08 08:54:23,795 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,795 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:23,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:23,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:23,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:23,795 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,795 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:23,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:23,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:23,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:23,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:23,796 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 08:54:23,796 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2a41feeb[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:23,796 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:23,797 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,798 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,798 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:23,798 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-08 08:54:23,798 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:23,798 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:23,798 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:23,800 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 08:54:23,800 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:23,800 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:23,800 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-08 08:54:23,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:23,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:23,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:23,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:23,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:23,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:23,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:23,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:23,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:23,802 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:23,802 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-08 08:54:23,802 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:23,802 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:23,802 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,803 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:23,804 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,804 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,804 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:23,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:23,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:23,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:23,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:23,805 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:23,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:23,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:23,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,806 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:23,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:23,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:23,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:23,808 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:23,808 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,808 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:23,808 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,808 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:23,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:23,808 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:23,808 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 08:54:23,808 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:23,808 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 08:54:23,808 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:23,808 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:23,808 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,808 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,808 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:23,808 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:23,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:23,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:23,909 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:23,909 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:23,909 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:23,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:23,913 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:23,913 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:23,914 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:23,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:23,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,915 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 08:54:23,915 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:23,915 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 08:54:23,915 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:23,916 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,916 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,919 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:23,919 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:23,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:23,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:23,919 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:23,919 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:23,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,920 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:23,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,920 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,920 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 08:54:23,920 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:23,920 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 08:54:23,920 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:23,920 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 08:54:23,921 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:23,921 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:23,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:23,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. started... 2023-02-08 08:54:23,921 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b7daddc3345a00410a8096f6715baffd 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:24,335 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228efecc0408b4c41fe87120586ac188905 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228efecc0408b4c41fe87120586ac188905 2023-02-08 08:54:24,585 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 08:54:24,741 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/088c3cc97c204cfc91b9303fb6f0a1dd 2023-02-08 08:54:24,745 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/088c3cc97c204cfc91b9303fb6f0a1dd as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/088c3cc97c204cfc91b9303fb6f0a1dd 2023-02-08 08:54:24,749 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/088c3cc97c204cfc91b9303fb6f0a1dd, entries=1, sequenceid=21, filesize=4.9 K 2023-02-08 08:54:24,750 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b7daddc3345a00410a8096f6715baffd in 829ms, sequenceid=21, compaction requested=true 2023-02-08 08:54:24,750 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b7daddc3345a00410a8096f6715baffd: 2023-02-08 08:54:24,750 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:24,750 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:24,750 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:24,750 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:24,750 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 08:54:24,750 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-08 08:54:24,756 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:24,756 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:24,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:24,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:24,757 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:24,757 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:24,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:24,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:24,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:24,758 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 08:54:24,758 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:24,758 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@707d9947[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:24,758 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,759 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,759 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:24,759 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-08 08:54:24,759 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:24,759 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:24,759 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:24,761 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 08:54:24,761 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:24,761 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:24,761 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-08 08:54:24,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:24,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:24,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:24,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:24,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:24,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:24,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:24,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:24,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:24,763 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:24,763 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-08 08:54:24,763 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:24,763 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:24,763 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,763 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:24,767 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,767 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:24,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:24,767 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:24,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,767 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:24,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:24,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:24,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:24,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:24,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,768 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:24,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:24,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:24,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:24,770 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:24,770 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:24,770 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,770 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 08:54:24,770 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:24,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:24,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:24,770 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:24,770 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:24,770 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 08:54:24,770 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:24,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:24,771 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:24,770 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:24,771 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:24,771 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,771 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,771 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,871 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:24,871 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:24,871 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:24,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:24,876 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:24,876 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:24,877 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:24,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:24,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,879 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 08:54:24,879 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:24,879 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 08:54:24,879 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:24,879 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,879 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,880 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:24,880 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:24,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:24,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:24,881 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:24,881 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:24,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,881 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:24,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,881 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,882 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 08:54:24,882 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:24,882 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 08:54:24,882 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:24,882 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 08:54:24,882 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:24,882 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:24,882 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:24,882 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. started... 2023-02-08 08:54:24,882 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b7daddc3345a00410a8096f6715baffd 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:25,295 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231bb5ffc9b4bfd46bbb84802abdd2ca6da to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231bb5ffc9b4bfd46bbb84802abdd2ca6da 2023-02-08 08:54:25,700 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/483091f11d0d435eb15497e45195f487 2023-02-08 08:54:25,706 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/483091f11d0d435eb15497e45195f487 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/483091f11d0d435eb15497e45195f487 2023-02-08 08:54:25,711 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/483091f11d0d435eb15497e45195f487, entries=1, sequenceid=25, filesize=4.9 K 2023-02-08 08:54:25,712 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b7daddc3345a00410a8096f6715baffd in 830ms, sequenceid=25, compaction requested=true 2023-02-08 08:54:25,713 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b7daddc3345a00410a8096f6715baffd: 2023-02-08 08:54:25,713 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:25,713 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:25,713 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:25,713 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,713 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 08:54:25,713 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-08 08:54:25,715 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,715 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:25,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:25,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:25,715 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,716 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:25,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:25,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:25,718 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 08:54:25,718 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3486263e[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:25,718 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:25,718 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,719 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,719 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:25,719 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-08 08:54:25,719 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:25,720 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,720 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:25,721 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,721 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 08:54:25,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:25,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:25,721 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:25,721 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-08 08:54:25,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:25,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:25,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:25,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:25,725 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:25,725 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-08 08:54:25,725 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:25,725 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:25,725 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,725 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:25,727 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,728 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:25,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:25,728 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:25,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:25,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:25,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:25,728 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:25,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,730 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:25,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,732 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,732 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:25,732 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:25,732 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,732 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,732 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:25,732 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:25,732 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 08:54:25,732 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:25,732 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:25,732 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,733 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,733 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:25,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:25,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:25,733 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:25,833 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:25,833 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:25,833 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:25,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:25,838 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:25,839 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:25,839 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:25,841 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:25,841 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,848 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 08:54:25,848 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:25,848 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 08:54:25,848 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:25,849 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,849 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,852 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,853 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,853 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:25,857 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:25,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:25,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:25,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:25,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,858 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,864 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 08:54:25,864 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,864 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 08:54:25,864 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:25,864 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 08:54:25,864 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:25,864 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:25,865 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. started... 2023-02-08 08:54:25,865 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b7daddc3345a00410a8096f6715baffd 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:25,864 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:25,892 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601014ffa157093154bbf8eafd9bb549c90ef to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601014ffa157093154bbf8eafd9bb549c90ef 2023-02-08 08:54:25,910 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/a24f27bcca5a40f4b6c20e610b738ef4 2023-02-08 08:54:25,917 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/a24f27bcca5a40f4b6c20e610b738ef4 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/a24f27bcca5a40f4b6c20e610b738ef4 2023-02-08 08:54:25,921 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/a24f27bcca5a40f4b6c20e610b738ef4, entries=1, sequenceid=29, filesize=4.9 K 2023-02-08 08:54:25,922 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b7daddc3345a00410a8096f6715baffd in 57ms, sequenceid=29, compaction requested=true 2023-02-08 08:54:25,922 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b7daddc3345a00410a8096f6715baffd: 2023-02-08 08:54:25,922 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:25,922 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:25,922 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:25,922 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,922 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 08:54:25,922 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-08 08:54:25,923 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,923 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:25,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:25,924 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,924 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:25,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:25,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:25,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:25,925 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 08:54:25,925 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b231f20[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:25,925 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:25,926 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,928 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,928 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:25,928 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-08 08:54:25,928 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:25,928 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,928 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:25,929 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,929 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 08:54:25,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,929 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:25,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:25,929 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-08 08:54:25,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:25,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:25,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:25,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:25,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:25,931 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:25,931 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-08 08:54:25,931 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:25,931 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:25,931 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,931 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:25,933 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,933 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,933 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:25,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:25,933 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:25,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,933 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:25,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:25,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:25,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:25,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,934 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:25,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,936 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,936 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,936 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,936 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:25,936 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:25,937 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 08:54:25,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:25,936 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:25,936 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:25,937 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 08:54:25,937 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:25,937 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:25,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:25,937 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,937 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:25,937 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:25,937 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:26,037 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:26,038 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:26,038 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:26,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:26,042 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:26,042 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:26,043 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:26,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:26,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,045 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 08:54:26,045 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:26,045 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 08:54:26,045 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:26,045 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,045 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,050 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:26,050 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,051 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:26,051 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:26,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,051 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:26,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,052 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,052 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 08:54:26,052 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,052 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 08:54:26,052 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:26,052 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 08:54:26,052 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:26,052 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:26,052 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:26,052 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. started... 2023-02-08 08:54:26,052 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b7daddc3345a00410a8096f6715baffd 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:26,465 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601030356f3129f83417ab966e58d717234b7 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601030356f3129f83417ab966e58d717234b7 2023-02-08 08:54:26,871 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/d5834949ccb047b585e5ceebdbf0bc97 2023-02-08 08:54:26,875 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/d5834949ccb047b585e5ceebdbf0bc97 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/d5834949ccb047b585e5ceebdbf0bc97 2023-02-08 08:54:26,879 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/d5834949ccb047b585e5ceebdbf0bc97, entries=1, sequenceid=33, filesize=4.9 K 2023-02-08 08:54:26,880 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b7daddc3345a00410a8096f6715baffd in 828ms, sequenceid=33, compaction requested=true 2023-02-08 08:54:26,880 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b7daddc3345a00410a8096f6715baffd: 2023-02-08 08:54:26,880 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:26,880 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:26,880 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:26,880 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:26,880 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 08:54:26,880 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-08 08:54:26,882 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,882 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:26,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:26,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:26,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:26,882 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,882 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:26,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:26,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:26,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:26,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:26,883 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 08:54:26,883 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e300c09[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:26,883 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:26,883 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,884 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,884 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:26,884 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-08 08:54:26,885 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:26,885 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:26,885 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:26,886 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 08:54:26,886 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:26,886 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:26,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:26,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:26,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:26,887 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-08 08:54:26,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:26,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:26,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:26,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:26,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:26,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:26,889 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:26,889 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-08 08:54:26,889 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:26,889 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:26,889 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,889 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:26,890 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,890 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:26,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:26,890 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,891 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:26,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:26,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:26,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,892 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:26,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:26,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:26,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:26,894 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:26,894 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:26,894 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:26,894 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:26,894 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:26,894 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 08:54:26,894 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:26,894 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 08:54:26,894 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:26,894 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:26,894 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,895 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,895 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:26,895 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:26,895 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:26,895 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:26,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:26,995 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:26,995 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:26,995 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:26,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:26,999 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:26,999 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:27,000 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:27,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:27,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,002 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 08:54:27,002 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,002 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 08:54:27,002 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:27,002 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,002 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,003 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:27,003 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,004 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:27,004 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:27,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,004 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:27,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,005 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,005 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 08:54:27,005 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,005 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 08:54:27,005 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,005 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 08:54:27,005 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:27,005 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:27,005 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:27,005 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. started... 2023-02-08 08:54:27,005 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b7daddc3345a00410a8096f6715baffd 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:27,418 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907d81d4454aa3f439690cf4ff8770796bf to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20150907d81d4454aa3f439690cf4ff8770796bf 2023-02-08 08:54:27,823 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/c845cdf0b7b94139ad24d64830a9b85c 2023-02-08 08:54:27,829 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/c845cdf0b7b94139ad24d64830a9b85c as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/c845cdf0b7b94139ad24d64830a9b85c 2023-02-08 08:54:27,833 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/c845cdf0b7b94139ad24d64830a9b85c, entries=1, sequenceid=37, filesize=4.9 K 2023-02-08 08:54:27,834 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b7daddc3345a00410a8096f6715baffd in 829ms, sequenceid=37, compaction requested=true 2023-02-08 08:54:27,834 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b7daddc3345a00410a8096f6715baffd: 2023-02-08 08:54:27,835 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:27,835 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:27,835 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:27,835 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:27,835 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 08:54:27,835 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-08 08:54:27,837 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:27,837 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:27,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:27,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:27,837 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,838 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:27,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:27,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:27,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:27,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:27,839 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 08:54:27,839 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:27,839 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@221b0d29[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:27,839 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,840 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,841 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:27,841 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-08 08:54:27,841 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:27,841 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:27,841 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:27,842 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 08:54:27,842 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:27,842 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:27,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:27,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:27,842 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-08 08:54:27,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:27,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:27,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:27,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:27,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:27,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:27,845 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:27,845 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-08 08:54:27,845 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:27,845 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:27,845 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,845 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:27,847 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,847 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,847 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:27,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:27,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:27,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,847 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:27,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:27,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,849 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:27,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:27,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:27,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:27,852 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:27,852 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,852 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,852 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:27,852 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 08:54:27,852 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,852 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 08:54:27,852 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:27,852 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:27,853 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:27,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,853 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,853 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,853 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,853 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:27,853 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:27,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:27,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:27,953 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:27,953 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:27,953 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:27,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:27,958 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:27,958 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:27,959 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:27,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:27,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,960 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 08:54:27,960 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,960 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-08 08:54:27,960 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-08 08:54:27,961 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,961 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,963 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:27,963 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:27,963 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:27,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:27,963 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:27,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,963 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:27,963 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:27,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,964 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,964 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-08 08:54:27,964 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:27,964 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-08 08:54:27,964 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:27,964 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-08 08:54:27,964 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:27,964 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:27,964 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:27,964 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. started... 2023-02-08 08:54:27,964 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing b7daddc3345a00410a8096f6715baffd 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:28,380 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511204908e5365ca34200bcbacd621e3d6c70 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511204908e5365ca34200bcbacd621e3d6c70 2023-02-08 08:54:28,786 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/9114d243bb424956b81aedd644ee6809 2023-02-08 08:54:28,791 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/9114d243bb424956b81aedd644ee6809 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/9114d243bb424956b81aedd644ee6809 2023-02-08 08:54:28,796 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/9114d243bb424956b81aedd644ee6809, entries=1, sequenceid=41, filesize=4.9 K 2023-02-08 08:54:28,797 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for b7daddc3345a00410a8096f6715baffd in 833ms, sequenceid=41, compaction requested=true 2023-02-08 08:54:28,797 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for b7daddc3345a00410a8096f6715baffd: 2023-02-08 08:54:28,797 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:28,797 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:28,797 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:28,797 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:28,798 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-08 08:54:28,798 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-08 08:54:28,799 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:28,799 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:28,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:28,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:28,800 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,800 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:28,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:28,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:28,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:28,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:28,801 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-08 08:54:28,801 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27296882[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:28,801 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-08 08:54:28,801 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,802 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,803 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-08 08:54:28,803 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-08 08:54:28,803 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:28,803 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:28,803 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:28,804 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-08 08:54:28,804 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:28,805 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:28,805 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-08 08:54:28,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:28,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:28,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:28,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:28,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:28,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:28,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:28,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:28,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:28,808 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:54:28,808 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-08 08:54:28,808 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:28,808 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:28,808 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,808 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:28,811 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,811 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:28,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,811 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:28,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:28,811 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:28,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:28,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:28,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:28,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:28,813 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:28,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:28,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:28,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:28,828 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:28,828 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:28,828 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:28,828 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-08 08:54:28,828 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:28,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:28,828 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:28,828 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,828 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,828 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:28,828 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,829 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,829 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:28,829 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:28,828 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-08 08:54:28,829 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:28,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:28,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:28,929 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:28,929 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:28,929 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:28,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:28,931 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37869] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-02-08 08:54:28,933 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 08:54:28,937 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 08:54:28,938 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675847068938 2023-02-08 08:54:28,938 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:54:28,942 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 08:54:28,943 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-02-08 08:54:28,943 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 08:54:28,943 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] zookeeper.ReadOnlyZKClient(139): Connect 0x7bd8efce to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:54:28,958 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3548dff1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:54:28,959 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 08:54:29,819 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ece541d2108a47148f073d3c3efe2901 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160103ece541d2108a47148f073d3c3efe2901 2023-02-08 08:54:29,820 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 08:54:29,833 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:54:29,834 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56830, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:54:29,840 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:29,840 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:29,841 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 08:54:29,842 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35004, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 08:54:29,851 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3667a132c215452590fb23712ac5fe7e first=Optional[r3] last=Optional[r6] 2023-02-08 08:54:29,851 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3667a132c215452590fb23712ac5fe7e}] 2023-02-08 08:54:29,855 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3667a132c215452590fb23712ac5fe7e for inclusion in b7daddc3345a00410a8096f6715baffd/family1 2023-02-08 08:54:29,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-08 08:54:29,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 08:54:29,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for b7daddc3345a00410a8096f6715baffd: 2023-02-08 08:54:29,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/3667a132c215452590fb23712ac5fe7e to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ejfbnr4ac6qrejr849igne6ov2vb7c3lmficp8hvqh0s0rgtrqcue6c84q6etn05/family1/3667a132c215452590fb23712ac5fe7e 2023-02-08 08:54:29,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ejfbnr4ac6qrejr849igne6ov2vb7c3lmficp8hvqh0s0rgtrqcue6c84q6etn05/family1/3667a132c215452590fb23712ac5fe7e as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/5e796a65e7a841399a9504d570b904d8_SeqId_44_ 2023-02-08 08:54:29,861 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ejfbnr4ac6qrejr849igne6ov2vb7c3lmficp8hvqh0s0rgtrqcue6c84q6etn05/family1/3667a132c215452590fb23712ac5fe7e into b7daddc3345a00410a8096f6715baffd/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/5e796a65e7a841399a9504d570b904d8_SeqId_44_ - updating store file list. 2023-02-08 08:54:29,865 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/5e796a65e7a841399a9504d570b904d8_SeqId_44_ into b7daddc3345a00410a8096f6715baffd/family1 2023-02-08 08:54:29,865 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ejfbnr4ac6qrejr849igne6ov2vb7c3lmficp8hvqh0s0rgtrqcue6c84q6etn05/family1/3667a132c215452590fb23712ac5fe7e into b7daddc3345a00410a8096f6715baffd/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/5e796a65e7a841399a9504d570b904d8_SeqId_44_) 2023-02-08 08:54:29,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyPolicy__ejfbnr4ac6qrejr849igne6ov2vb7c3lmficp8hvqh0s0rgtrqcue6c84q6etn05/family1/3667a132c215452590fb23712ac5fe7e 2023-02-08 08:54:29,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-08 08:54:29,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-08 08:54:29,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(1912): b7daddc3345a00410a8096f6715baffd/family1 is initiating minor compaction 2023-02-08 08:54:29,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7e32977b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:29,868 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b7daddc3345a00410a8096f6715baffd/family1 in testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:29,869 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/506f31968af04443b20c031965715602, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/3edd5addbded4b4a9fbef04b8f4d1c30, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/3948e90b496647b0999bc0d714ecd381, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/088c3cc97c204cfc91b9303fb6f0a1dd, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/483091f11d0d435eb15497e45195f487, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/a24f27bcca5a40f4b6c20e610b738ef4, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/d5834949ccb047b585e5ceebdbf0bc97, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/c845cdf0b7b94139ad24d64830a9b85c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/9114d243bb424956b81aedd644ee6809, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/5e796a65e7a841399a9504d570b904d8_SeqId_44_] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp, totalSize=50.0 K 2023-02-08 08:54:29,869 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 506f31968af04443b20c031965715602, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-08 08:54:29,869 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 3edd5addbded4b4a9fbef04b8f4d1c30, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-08 08:54:29,869 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 3948e90b496647b0999bc0d714ecd381, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-08 08:54:29,869 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 088c3cc97c204cfc91b9303fb6f0a1dd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-08 08:54:29,869 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 483091f11d0d435eb15497e45195f487, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-08 08:54:29,869 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting a24f27bcca5a40f4b6c20e610b738ef4, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-08 08:54:29,869 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting d5834949ccb047b585e5ceebdbf0bc97, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-08 08:54:29,869 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting c845cdf0b7b94139ad24d64830a9b85c, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-08 08:54:29,869 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 9114d243bb424956b81aedd644ee6809, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-08 08:54:29,869 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 5e796a65e7a841399a9504d570b904d8_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-08 08:54:29,870 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 08:54:29,871 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:29,872 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228efecc0408b4c41fe87120586ac188905 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151228efecc0408b4c41fe87120586ac188905 2023-02-08 08:54:29,873 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231bb5ffc9b4bfd46bbb84802abdd2ca6da to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151231bb5ffc9b4bfd46bbb84802abdd2ca6da 2023-02-08 08:54:29,875 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601014ffa157093154bbf8eafd9bb549c90ef to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601014ffa157093154bbf8eafd9bb549c90ef 2023-02-08 08:54:29,876 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601030356f3129f83417ab966e58d717234b7 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601030356f3129f83417ab966e58d717234b7 2023-02-08 08:54:29,876 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-02-08 08:54:29,876 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-08 08:54:29,877 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 08:54:29,877 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 08:54:29,877 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 08:54:29,877 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 08:54:29,877 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 08:54:29,877 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 08:54:29,877 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 08:54:29,877 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 08:54:29,904 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] throttle.PressureAwareThroughputController(145): b7daddc3345a00410a8096f6715baffd#family1#compaction#48 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:30,720 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/1af977c86f454c898f565153e0fa4ffa as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/1af977c86f454c898f565153e0fa4ffa 2023-02-08 08:54:30,725 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in b7daddc3345a00410a8096f6715baffd/family1 of b7daddc3345a00410a8096f6715baffd into 1af977c86f454c898f565153e0fa4ffa(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:30,725 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b7daddc3345a00410a8096f6715baffd: 2023-02-08 08:54:30,725 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1675846469868; duration=0sec 2023-02-08 08:54:30,725 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:30,730 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512050847043d6f41439db190863589e605e7 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512050847043d6f41439db190863589e605e7 2023-02-08 08:54:30,731 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 08:54:30,747 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:30,747 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:30,755 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1bfc9f173b514f48822dea6e01aa078b first=Optional[r00] last=Optional[r2] 2023-02-08 08:54:30,755 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1bfc9f173b514f48822dea6e01aa078b}] 2023-02-08 08:54:30,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1bfc9f173b514f48822dea6e01aa078b for inclusion in b7daddc3345a00410a8096f6715baffd/family1 2023-02-08 08:54:30,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-08 08:54:30,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 08:54:30,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for b7daddc3345a00410a8096f6715baffd: 2023-02-08 08:54:30,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1bfc9f173b514f48822dea6e01aa078b to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyPolicy__bsd9rsvm1682h8n1329sjttukogltu4smmlan5fjsd0ta09ibcohvjruke6nigu5/family1/1bfc9f173b514f48822dea6e01aa078b 2023-02-08 08:54:30,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyPolicy__bsd9rsvm1682h8n1329sjttukogltu4smmlan5fjsd0ta09ibcohvjruke6nigu5/family1/1bfc9f173b514f48822dea6e01aa078b as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/0dc8dc8ed7aa40f289fb8e472b6047d5_SeqId_47_ 2023-02-08 08:54:30,764 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyPolicy__bsd9rsvm1682h8n1329sjttukogltu4smmlan5fjsd0ta09ibcohvjruke6nigu5/family1/1bfc9f173b514f48822dea6e01aa078b into b7daddc3345a00410a8096f6715baffd/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/0dc8dc8ed7aa40f289fb8e472b6047d5_SeqId_47_ - updating store file list. 2023-02-08 08:54:30,768 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/0dc8dc8ed7aa40f289fb8e472b6047d5_SeqId_47_ into b7daddc3345a00410a8096f6715baffd/family1 2023-02-08 08:54:30,768 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyPolicy__bsd9rsvm1682h8n1329sjttukogltu4smmlan5fjsd0ta09ibcohvjruke6nigu5/family1/1bfc9f173b514f48822dea6e01aa078b into b7daddc3345a00410a8096f6715baffd/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/0dc8dc8ed7aa40f289fb8e472b6047d5_SeqId_47_) 2023-02-08 08:54:30,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyPolicy__bsd9rsvm1682h8n1329sjttukogltu4smmlan5fjsd0ta09ibcohvjruke6nigu5/family1/1bfc9f173b514f48822dea6e01aa078b 2023-02-08 08:54:30,769 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-08 08:54:30,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 345120 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-08 08:54:30,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(1912): b7daddc3345a00410a8096f6715baffd/family1 is initiating minor compaction (all files) 2023-02-08 08:54:30,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@17cd0db4; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:30,770 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of b7daddc3345a00410a8096f6715baffd/family1 in testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:54:30,770 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/a4061aaf1fb44546a4d35af015c732ed, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/1af977c86f454c898f565153e0fa4ffa, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/0dc8dc8ed7aa40f289fb8e472b6047d5_SeqId_47_] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp, totalSize=337.0 K 2023-02-08 08:54:30,770 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting a4061aaf1fb44546a4d35af015c732ed, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878836535 2023-02-08 08:54:30,771 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 1af977c86f454c898f565153e0fa4ffa, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621236535 2023-02-08 08:54:30,771 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 0dc8dc8ed7aa40f289fb8e472b6047d5_SeqId_47_, keycount=1002, bloomtype=ROW, size=165.4 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878836535 2023-02-08 08:54:30,772 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 08:54:30,773 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:30,774 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511306080048901c24e1097688267690bdd71 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201511306080048901c24e1097688267690bdd71 2023-02-08 08:54:30,775 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201da734f8416464deb9104fb351eee7edb to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151201da734f8416464deb9104fb351eee7edb 2023-02-08 08:54:30,777 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205c756982438de4f97bebd3aa2a2a0cf2e to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151205c756982438de4f97bebd3aa2a2a0cf2e 2023-02-08 08:54:30,777 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-02-08 08:54:30,777 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-08 08:54:30,777 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-08 08:54:30,777 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-08 08:54:30,777 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:54:30,778 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 08:54:30,780 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.8470 sec 2023-02-08 08:54:30,787 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] throttle.PressureAwareThroughputController(145): b7daddc3345a00410a8096f6715baffd#family1#compaction#49 average throughput is 19.33 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:54:30,842 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/506f31968af04443b20c031965715602, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/3edd5addbded4b4a9fbef04b8f4d1c30, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/3948e90b496647b0999bc0d714ecd381, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/088c3cc97c204cfc91b9303fb6f0a1dd, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/483091f11d0d435eb15497e45195f487, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/a24f27bcca5a40f4b6c20e610b738ef4, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/d5834949ccb047b585e5ceebdbf0bc97, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/c845cdf0b7b94139ad24d64830a9b85c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/9114d243bb424956b81aedd644ee6809, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/5e796a65e7a841399a9504d570b904d8_SeqId_44_] to archive 2023-02-08 08:54:30,843 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:30,845 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/506f31968af04443b20c031965715602 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/506f31968af04443b20c031965715602 2023-02-08 08:54:30,847 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/3edd5addbded4b4a9fbef04b8f4d1c30 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/3edd5addbded4b4a9fbef04b8f4d1c30 2023-02-08 08:54:30,849 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/3948e90b496647b0999bc0d714ecd381 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/3948e90b496647b0999bc0d714ecd381 2023-02-08 08:54:30,851 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/088c3cc97c204cfc91b9303fb6f0a1dd to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/088c3cc97c204cfc91b9303fb6f0a1dd 2023-02-08 08:54:30,853 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/483091f11d0d435eb15497e45195f487 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/483091f11d0d435eb15497e45195f487 2023-02-08 08:54:30,861 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/a24f27bcca5a40f4b6c20e610b738ef4 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/a24f27bcca5a40f4b6c20e610b738ef4 2023-02-08 08:54:30,863 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/d5834949ccb047b585e5ceebdbf0bc97 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/d5834949ccb047b585e5ceebdbf0bc97 2023-02-08 08:54:30,865 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/c845cdf0b7b94139ad24d64830a9b85c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/c845cdf0b7b94139ad24d64830a9b85c 2023-02-08 08:54:30,867 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/9114d243bb424956b81aedd644ee6809 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/9114d243bb424956b81aedd644ee6809 2023-02-08 08:54:30,869 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/5e796a65e7a841399a9504d570b904d8_SeqId_44_ to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/5e796a65e7a841399a9504d570b904d8_SeqId_44_ 2023-02-08 08:54:30,894 INFO [Listener at localhost/40763] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=263 (was 251) - Thread LEAK? -, OpenFileDescriptor=629 (was 600) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=328 (was 321) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=4225 (was 4333) 2023-02-08 08:54:30,905 INFO [Listener at localhost/40763] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=263, OpenFileDescriptor=629, MaxFileDescriptor=60000, SystemLoadAverage=328, ProcessCount=168, AvailableMemoryMB=4225 2023-02-08 08:54:30,906 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:30,906 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:30,907 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-08 08:54:30,908 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:30,910 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:54:30,910 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-02-08 08:54:30,910 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:30,910 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:30,911 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:54:30,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-08 08:54:30,912 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:30,912 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e empty. 2023-02-08 08:54:30,913 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:30,913 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-02-08 08:54:31,011 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:31,012 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:31,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-08 08:54:31,213 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:31,213 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:31,213 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-08 08:54:31,320 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-02-08 08:54:31,322 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7671): creating {ENCODED => 370d7626b49b9b3c93d59dd5a186a23e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp 2023-02-08 08:54:31,514 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:31,514 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:31,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-08 08:54:31,727 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:31,728 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1603): Closing 370d7626b49b9b3c93d59dd5a186a23e, disabling compactions & flushes 2023-02-08 08:54:31,728 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:31,728 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:31,728 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. after waiting 0 ms 2023-02-08 08:54:31,728 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:31,728 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:31,728 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1557): Region close journal for 370d7626b49b9b3c93d59dd5a186a23e: Waiting for close lock at 1675846471728Disabling compacts and flushes for region at 1675846471728Disabling writes for close at 1675846471728Writing region close event to WAL at 1675846471728Closed at 1675846471728 2023-02-08 08:54:31,732 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:54:31,732 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675846471732"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846471732"}]},"ts":"1675846471732"} 2023-02-08 08:54:31,734 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 08:54:31,735 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:54:31,735 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846471735"}]},"ts":"1675846471735"} 2023-02-08 08:54:31,736 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-02-08 08:54:31,739 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=370d7626b49b9b3c93d59dd5a186a23e, ASSIGN}] 2023-02-08 08:54:31,740 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=370d7626b49b9b3c93d59dd5a186a23e, ASSIGN 2023-02-08 08:54:31,741 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=370d7626b49b9b3c93d59dd5a186a23e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=false 2023-02-08 08:54:31,891 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=370d7626b49b9b3c93d59dd5a186a23e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:31,892 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675846471891"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846471891"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846471891"}]},"ts":"1675846471891"} 2023-02-08 08:54:31,893 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure 370d7626b49b9b3c93d59dd5a186a23e, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:54:32,005 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/.tmp/family1/ee1aba2b6d2149a48d3b407dc5608d05 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/ee1aba2b6d2149a48d3b407dc5608d05 2023-02-08 08:54:32,010 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in b7daddc3345a00410a8096f6715baffd/family1 of b7daddc3345a00410a8096f6715baffd into ee1aba2b6d2149a48d3b407dc5608d05(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 08:54:32,010 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for b7daddc3345a00410a8096f6715baffd: 2023-02-08 08:54:32,010 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd., storeName=family1, fileCount=3, fileSize=337.0 K (165.1 K, 6.6 K, 165.4 K), priority=2, time=1675846470770; duration=1sec 2023-02-08 08:54:32,010 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:32,015 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:32,015 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:32,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-08 08:54:32,025 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/a4061aaf1fb44546a4d35af015c732ed, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/1af977c86f454c898f565153e0fa4ffa, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/0dc8dc8ed7aa40f289fb8e472b6047d5_SeqId_47_] to archive 2023-02-08 08:54:32,026 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:32,028 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/a4061aaf1fb44546a4d35af015c732ed to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/a4061aaf1fb44546a4d35af015c732ed 2023-02-08 08:54:32,029 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/1af977c86f454c898f565153e0fa4ffa to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/1af977c86f454c898f565153e0fa4ffa 2023-02-08 08:54:32,030 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/0dc8dc8ed7aa40f289fb8e472b6047d5_SeqId_47_ to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/family1/0dc8dc8ed7aa40f289fb8e472b6047d5_SeqId_47_ 2023-02-08 08:54:32,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:32,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 370d7626b49b9b3c93d59dd5a186a23e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:54:32,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:32,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:32,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:32,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:32,049 INFO [StoreOpener-370d7626b49b9b3c93d59dd5a186a23e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:32,051 DEBUG [StoreOpener-370d7626b49b9b3c93d59dd5a186a23e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1 2023-02-08 08:54:32,051 DEBUG [StoreOpener-370d7626b49b9b3c93d59dd5a186a23e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1 2023-02-08 08:54:32,051 INFO [StoreOpener-370d7626b49b9b3c93d59dd5a186a23e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 370d7626b49b9b3c93d59dd5a186a23e columnFamilyName family1 2023-02-08 08:54:32,052 INFO [StoreOpener-370d7626b49b9b3c93d59dd5a186a23e-1] regionserver.HStore(310): Store=370d7626b49b9b3c93d59dd5a186a23e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:32,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:32,054 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:32,056 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:32,057 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:54:32,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 370d7626b49b9b3c93d59dd5a186a23e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60936639, jitterRate=-0.09197331964969635}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 08:54:32,058 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 370d7626b49b9b3c93d59dd5a186a23e: Running coprocessor pre-open hook at 1675846472048Writing region info on filesystem at 1675846472048Initializing all the Stores at 1675846472049 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675846472049Cleaning up temporary data from old regions at 1675846472054 (+5 ms)Cleaning up detritus from prior splits at 1675846472054Running coprocessor post-open hooks at 1675846472058 (+4 ms)Region opened successfully at 1675846472058 2023-02-08 08:54:32,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e., pid=41, masterSystemTime=1675846472045 2023-02-08 08:54:32,060 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:32,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:32,060 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=370d7626b49b9b3c93d59dd5a186a23e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:32,060 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675846472060"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846472060"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846472060"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846472060"}]},"ts":"1675846472060"} 2023-02-08 08:54:32,063 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-02-08 08:54:32,063 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure 370d7626b49b9b3c93d59dd5a186a23e, server=jenkins-hbase4.apache.org,42239,1675846413348 in 169 msec 2023-02-08 08:54:32,065 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-02-08 08:54:32,065 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=370d7626b49b9b3c93d59dd5a186a23e, ASSIGN in 325 msec 2023-02-08 08:54:32,065 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:54:32,065 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846472065"}]},"ts":"1675846472065"} 2023-02-08 08:54:32,066 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-02-08 08:54:32,069 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:54:32,070 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1610 sec 2023-02-08 08:54:33,016 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:33,016 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:33,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-08 08:54:33,017 INFO [Listener at localhost/40763] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-02-08 08:54:33,041 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:33,041 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:33,042 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:33,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,044 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 08:54:33,044 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:33,044 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-08 08:54:33,044 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-08 08:54:33,044 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,045 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,047 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:33,047 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:33,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:33,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:33,047 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:33,047 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:33,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,048 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:33,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,048 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,048 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,048 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-08 08:54:33,048 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:33,048 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-08 08:54:33,049 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:33,049 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:33,049 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:33,049 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. started... 2023-02-08 08:54:33,049 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 370d7626b49b9b3c93d59dd5a186a23e 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-08 08:54:33,469 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511303fad8d9c2ca841c88f271439b98f84e8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511303fad8d9c2ca841c88f271439b98f84e8 2023-02-08 08:54:33,873 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/fa2a6f1e4e6d46529f9cdf442822ef0e 2023-02-08 08:54:33,877 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/fa2a6f1e4e6d46529f9cdf442822ef0e as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/fa2a6f1e4e6d46529f9cdf442822ef0e 2023-02-08 08:54:33,881 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/fa2a6f1e4e6d46529f9cdf442822ef0e, entries=1000, sequenceid=5, filesize=191.4 K 2023-02-08 08:54:33,882 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 370d7626b49b9b3c93d59dd5a186a23e in 833ms, sequenceid=5, compaction requested=false 2023-02-08 08:54:33,882 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 08:54:33,882 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 370d7626b49b9b3c93d59dd5a186a23e: 2023-02-08 08:54:33,882 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:33,882 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:33,882 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:33,883 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:33,883 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-08 08:54:33,883 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-08 08:54:33,884 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:33,884 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:33,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:33,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:33,884 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,884 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:33,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:33,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:33,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:33,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:33,886 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-08 08:54:33,886 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24f0b958[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:33,886 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-08 08:54:33,886 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,887 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,887 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-08 08:54:33,887 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-08 08:54:33,887 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:33,887 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:33,887 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:33,889 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-08 08:54:33,889 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:33,889 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:33,889 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-08 08:54:33,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:33,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:33,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:33,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:33,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:33,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:33,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:33,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:33,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:33,891 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-08 08:54:33,891 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-08 08:54:33,891 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:33,891 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:33,891 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,891 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:33,894 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,894 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:33,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:33,894 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:33,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:33,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:33,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:33,894 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:33,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:33,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,895 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:33,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:33,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:33,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:33,897 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:33,897 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,897 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,897 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:33,897 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:33,897 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:33,897 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:33,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:33,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:33,897 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 08:54:33,897 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:33,897 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-08 08:54:33,897 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,898 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,898 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:33,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:33,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:33,898 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:33,898 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:33,998 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:33,998 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:33,998 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:33,999 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:34,002 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:34,003 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:34,003 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:34,005 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:34,005 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,005 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 08:54:34,005 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:34,005 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-08 08:54:34,005 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-08 08:54:34,005 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,005 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,007 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:34,007 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:34,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:34,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:34,007 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:34,007 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:34,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,007 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:34,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,008 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,008 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-08 08:54:34,008 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,008 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-08 08:54:34,008 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:34,008 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-08 08:54:34,008 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:34,009 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:34,009 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:34,009 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. started... 2023-02-08 08:54:34,009 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 370d7626b49b9b3c93d59dd5a186a23e 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-08 08:54:34,422 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112881b190420da8413aae0e777bdfbfa706 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112881b190420da8413aae0e777bdfbfa706 2023-02-08 08:54:34,828 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/9616c3704c094ad8a8f79ad0669caed4 2023-02-08 08:54:34,834 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/9616c3704c094ad8a8f79ad0669caed4 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/9616c3704c094ad8a8f79ad0669caed4 2023-02-08 08:54:34,837 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/9616c3704c094ad8a8f79ad0669caed4, entries=1, sequenceid=9, filesize=5.0 K 2023-02-08 08:54:34,838 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 370d7626b49b9b3c93d59dd5a186a23e in 829ms, sequenceid=9, compaction requested=false 2023-02-08 08:54:34,839 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 370d7626b49b9b3c93d59dd5a186a23e: 2023-02-08 08:54:34,839 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:34,839 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:34,839 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:34,839 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:34,839 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-08 08:54:34,839 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-08 08:54:34,841 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,841 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:34,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:34,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:34,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:34,841 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,841 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:34,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:34,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:34,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:34,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:34,843 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-08 08:54:34,843 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a9eb98b[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:34,843 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-08 08:54:34,843 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,844 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,844 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-08 08:54:34,844 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-08 08:54:34,844 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:34,844 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:34,844 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:34,846 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-08 08:54:34,846 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:34,846 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:34,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:34,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:34,846 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-08 08:54:34,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:34,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:34,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:34,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:34,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:34,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:34,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:34,848 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-08 08:54:34,848 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-08 08:54:34,848 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:34,848 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:34,848 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,848 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:34,851 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,851 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:34,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:34,851 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:34,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:34,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:34,851 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:34,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:34,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:34,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,852 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:34,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:34,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:34,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:34,858 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:34,858 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:34,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:34,858 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:34,858 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 08:54:34,858 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:34,858 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:34,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-08 08:54:34,858 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:34,858 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,858 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:34,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:34,859 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:34,859 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,859 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:34,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:34,859 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,859 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,959 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:34,959 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:34,959 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:34,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:34,963 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:34,963 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:34,964 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:34,965 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:34,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,966 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 08:54:34,966 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:34,966 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-08 08:54:34,966 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-08 08:54:34,966 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,966 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,968 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:34,968 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:34,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:34,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:34,968 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:34,968 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:34,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,968 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:34,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,969 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,969 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-08 08:54:34,969 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:34,969 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-08 08:54:34,969 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:34,969 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-08 08:54:34,969 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:34,969 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:34,969 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:34,969 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. started... 2023-02-08 08:54:34,969 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 370d7626b49b9b3c93d59dd5a186a23e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:35,383 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512014cb39f28925449cf88565065b766a235 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512014cb39f28925449cf88565065b766a235 2023-02-08 08:54:35,789 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/de16d57403b34f4bbb9740c85ffef7ab 2023-02-08 08:54:35,794 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/de16d57403b34f4bbb9740c85ffef7ab as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/de16d57403b34f4bbb9740c85ffef7ab 2023-02-08 08:54:35,798 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/de16d57403b34f4bbb9740c85ffef7ab, entries=1, sequenceid=13, filesize=5.0 K 2023-02-08 08:54:35,799 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 370d7626b49b9b3c93d59dd5a186a23e in 830ms, sequenceid=13, compaction requested=true 2023-02-08 08:54:35,799 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 370d7626b49b9b3c93d59dd5a186a23e: 2023-02-08 08:54:35,799 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:35,799 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:35,799 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:35,799 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:35,799 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-08 08:54:35,799 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-08 08:54:35,802 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:35,802 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:35,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:35,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:35,802 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,803 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:35,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:35,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:35,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:35,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:35,804 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-08 08:54:35,804 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60169db4[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:35,804 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-08 08:54:35,804 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,805 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,805 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-08 08:54:35,805 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-08 08:54:35,805 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:35,805 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:35,805 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:35,806 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:35,806 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-08 08:54:35,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:35,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:35,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:35,806 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:35,807 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-08 08:54:35,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:35,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:35,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:35,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:35,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:35,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:35,809 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-08 08:54:35,809 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-08 08:54:35,809 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:35,809 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:35,809 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,809 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:35,810 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,810 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:35,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:35,810 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:35,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:35,811 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:35,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:35,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:35,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:35,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,812 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:35,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:35,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:35,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:35,815 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:35,815 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:35,815 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:35,815 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:35,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:35,815 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:35,815 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-08 08:54:35,815 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 08:54:35,815 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:35,815 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,815 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:35,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:35,815 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:35,815 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:35,815 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,816 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:35,816 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,916 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:35,916 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:35,916 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:35,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:35,920 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:35,920 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:35,921 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:35,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:35,922 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,923 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 08:54:35,923 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:35,923 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-08 08:54:35,923 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-08 08:54:35,923 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,923 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,924 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:35,924 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:35,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:35,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:35,924 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:35,924 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:35,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,925 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:35,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,925 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,925 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-08 08:54:35,925 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:35,925 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-08 08:54:35,925 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:35,925 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-08 08:54:35,926 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:35,926 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:35,926 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:35,926 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. started... 2023-02-08 08:54:35,926 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 370d7626b49b9b3c93d59dd5a186a23e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:36,338 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120539b877a8167d4afb8da03e8c02d1c57f to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120539b877a8167d4afb8da03e8c02d1c57f 2023-02-08 08:54:36,744 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/861892cf416142ef8dbcef24da775976 2023-02-08 08:54:36,748 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/861892cf416142ef8dbcef24da775976 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/861892cf416142ef8dbcef24da775976 2023-02-08 08:54:36,752 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/861892cf416142ef8dbcef24da775976, entries=1, sequenceid=17, filesize=5.0 K 2023-02-08 08:54:36,753 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 370d7626b49b9b3c93d59dd5a186a23e in 827ms, sequenceid=17, compaction requested=true 2023-02-08 08:54:36,753 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 370d7626b49b9b3c93d59dd5a186a23e: 2023-02-08 08:54:36,753 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:36,754 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:36,754 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:36,754 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:36,754 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-08 08:54:36,754 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-08 08:54:36,755 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,755 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:36,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:36,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:36,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:36,755 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,755 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:36,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:36,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:36,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:36,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:36,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-08 08:54:36,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@c97844c[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:36,757 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-08 08:54:36,757 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,758 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,758 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-08 08:54:36,758 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-08 08:54:36,758 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:36,758 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:36,758 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:36,761 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:36,761 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-08 08:54:36,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:36,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:36,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:36,761 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:36,761 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-08 08:54:36,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:36,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:36,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:36,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:36,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:36,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:36,763 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-08 08:54:36,763 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-08 08:54:36,763 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:36,763 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:36,763 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,763 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:36,764 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,764 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,764 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:36,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:36,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:36,765 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:36,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:36,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:36,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:36,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:36,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,766 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:36,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:36,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:36,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:36,768 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:36,768 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,768 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,768 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:36,768 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 08:54:36,768 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:36,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:36,768 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:36,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:36,768 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-08 08:54:36,768 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:36,768 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:36,768 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:36,768 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:36,768 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:36,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:36,768 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,769 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,869 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:36,869 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:36,869 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:36,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:36,873 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:36,873 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:36,874 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:36,896 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 08:54:36,896 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7706c0e8 to 127.0.0.1:61594 2023-02-08 08:54:36,896 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:54:36,899 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 08:54:36,899 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x38aeb583 to 127.0.0.1:61594 2023-02-08 08:54:36,899 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:54:36,900 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 08:54:36,900 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7bd8efce to 127.0.0.1:61594 2023-02-08 08:54:36,900 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:54:36,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:36,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,903 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 08:54:36,903 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:36,903 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-08 08:54:36,903 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-08 08:54:36,906 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,906 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,907 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:36,907 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:36,907 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:36,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:36,907 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:36,907 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:36,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,907 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:36,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,908 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,908 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:36,908 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-08 08:54:36,908 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:36,908 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-08 08:54:36,908 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:36,908 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:36,908 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:36,908 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. started... 2023-02-08 08:54:36,908 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 370d7626b49b9b3c93d59dd5a186a23e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:37,322 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512283ba9d43649724e99b8003cff978c3f12 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512283ba9d43649724e99b8003cff978c3f12 2023-02-08 08:54:37,576 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 08:54:37,728 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/5a229c63b0f144ccbdb29aab4ec9ae42 2023-02-08 08:54:37,732 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/5a229c63b0f144ccbdb29aab4ec9ae42 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/5a229c63b0f144ccbdb29aab4ec9ae42 2023-02-08 08:54:37,737 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/5a229c63b0f144ccbdb29aab4ec9ae42, entries=1, sequenceid=21, filesize=5.0 K 2023-02-08 08:54:37,738 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 370d7626b49b9b3c93d59dd5a186a23e in 830ms, sequenceid=21, compaction requested=true 2023-02-08 08:54:37,738 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 370d7626b49b9b3c93d59dd5a186a23e: 2023-02-08 08:54:37,738 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:37,738 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:37,738 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:37,738 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:37,738 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-08 08:54:37,738 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-08 08:54:37,744 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:37,744 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:37,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:37,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:37,745 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,745 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:37,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:37,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:37,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:37,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:37,746 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-08 08:54:37,746 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2d8cd6ff[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:37,746 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-08 08:54:37,746 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,747 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,747 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-08 08:54:37,747 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-08 08:54:37,747 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:37,747 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:37,747 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:37,749 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:37,749 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-08 08:54:37,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:37,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:37,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:37,749 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:37,749 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-08 08:54:37,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:37,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:37,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:37,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:37,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:37,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:37,751 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-08 08:54:37,751 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-08 08:54:37,751 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:37,751 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:37,751 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,751 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:37,754 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,754 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,754 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:37,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:37,754 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:37,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:37,755 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:37,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:37,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:37,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:37,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:37,756 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:37,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:37,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:37,758 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:37,758 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,758 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:37,758 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:37,758 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,758 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:37,758 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,758 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 08:54:37,758 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:37,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:37,758 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-08 08:54:37,758 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,758 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:37,758 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:37,758 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:37,759 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:37,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:37,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:37,859 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:37,859 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:37,859 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:37,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:37,863 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:37,863 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:37,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:37,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:37,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,865 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 08:54:37,865 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:37,865 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-08 08:54:37,865 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-08 08:54:37,866 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,866 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,867 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:37,867 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:37,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:37,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:37,867 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:37,867 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:37,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,868 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:37,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,868 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,868 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-08 08:54:37,868 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:37,868 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-08 08:54:37,868 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:37,868 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-08 08:54:37,868 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:37,869 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:37,869 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:37,869 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. started... 2023-02-08 08:54:37,869 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 370d7626b49b9b3c93d59dd5a186a23e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:38,281 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231754e782c24b4492c974d5bcf56866c25 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231754e782c24b4492c974d5bcf56866c25 2023-02-08 08:54:38,686 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/d029723c5c114d87b033577885af827b 2023-02-08 08:54:38,690 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/d029723c5c114d87b033577885af827b as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/d029723c5c114d87b033577885af827b 2023-02-08 08:54:38,694 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/d029723c5c114d87b033577885af827b, entries=1, sequenceid=25, filesize=5.0 K 2023-02-08 08:54:38,695 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 370d7626b49b9b3c93d59dd5a186a23e in 826ms, sequenceid=25, compaction requested=true 2023-02-08 08:54:38,695 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 370d7626b49b9b3c93d59dd5a186a23e: 2023-02-08 08:54:38,695 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:38,695 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:38,695 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:38,695 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:38,695 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-08 08:54:38,695 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-08 08:54:38,697 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:38,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:38,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:38,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:38,697 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:38,697 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,697 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:38,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:38,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:38,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:38,698 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-08 08:54:38,698 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@42b17024[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:38,698 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-08 08:54:38,698 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,700 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,700 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-08 08:54:38,701 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-08 08:54:38,701 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:38,701 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:38,701 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:38,702 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-08 08:54:38,702 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:38,702 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:38,702 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-08 08:54:38,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:38,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:38,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:38,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:38,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:38,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:38,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:38,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:38,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:38,704 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-08 08:54:38,704 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-08 08:54:38,704 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:38,704 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:38,704 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,704 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:38,705 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,705 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,705 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:38,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:38,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:38,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:38,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:38,706 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:38,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:38,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:38,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,707 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:38,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:38,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:38,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:38,708 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:38,708 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:38,709 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:38,709 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 08:54:38,709 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:38,709 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-08 08:54:38,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:38,709 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:38,709 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,709 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:38,709 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:38,709 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:38,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:38,709 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,709 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:38,709 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:38,709 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,809 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:38,809 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:38,809 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:38,810 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:38,814 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:38,814 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:38,814 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:38,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:38,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,816 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 08:54:38,816 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:38,816 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-08 08:54:38,816 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-08 08:54:38,816 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,817 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,818 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:38,818 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:38,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:38,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:38,819 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:38,819 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:38,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:38,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,819 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,819 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-08 08:54:38,820 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:38,820 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-08 08:54:38,820 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:38,820 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-08 08:54:38,820 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:38,820 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:38,820 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:38,820 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. started... 2023-02-08 08:54:38,820 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 370d7626b49b9b3c93d59dd5a186a23e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:39,232 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601019746410401e24123bf9a53573b408ca4 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601019746410401e24123bf9a53573b408ca4 2023-02-08 08:54:39,637 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/ab83dcf35cb04506bf753f1141eaa9f2 2023-02-08 08:54:39,641 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/ab83dcf35cb04506bf753f1141eaa9f2 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/ab83dcf35cb04506bf753f1141eaa9f2 2023-02-08 08:54:39,645 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/ab83dcf35cb04506bf753f1141eaa9f2, entries=1, sequenceid=29, filesize=5.0 K 2023-02-08 08:54:39,646 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 370d7626b49b9b3c93d59dd5a186a23e in 826ms, sequenceid=29, compaction requested=true 2023-02-08 08:54:39,646 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 370d7626b49b9b3c93d59dd5a186a23e: 2023-02-08 08:54:39,646 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:39,646 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:39,646 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:39,646 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:39,646 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-08 08:54:39,646 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-08 08:54:39,648 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,648 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:39,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:39,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:39,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:39,648 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,648 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:39,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:39,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:39,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:39,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:39,649 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-08 08:54:39,649 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-08 08:54:39,649 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@169eaefb[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:39,649 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,650 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,651 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-08 08:54:39,651 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-08 08:54:39,651 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:39,651 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:39,651 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:39,652 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:39,652 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-08 08:54:39,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:39,652 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:39,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:39,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:39,652 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-08 08:54:39,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:39,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:39,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:39,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:39,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:39,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:39,654 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-08 08:54:39,654 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-08 08:54:39,654 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:39,654 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:39,654 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,654 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:39,659 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,660 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:39,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:39,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,660 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:39,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,660 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:39,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:39,660 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:39,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:39,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:39,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,661 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:39,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:39,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:39,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:39,663 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:39,663 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,663 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:39,663 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,663 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 08:54:39,663 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:39,663 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:39,663 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,663 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,663 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,663 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-08 08:54:39,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:39,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:39,663 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:39,663 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:39,663 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:39,664 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:39,664 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:39,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:39,764 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:39,764 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:39,764 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:39,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:39,768 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:39,768 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:39,768 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:39,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:39,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,770 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 08:54:39,770 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:39,770 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-08 08:54:39,770 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-08 08:54:39,771 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,771 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,772 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:39,772 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:39,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:39,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:39,772 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:39,772 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:39,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,773 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:39,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,773 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,773 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-08 08:54:39,773 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:39,773 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-08 08:54:39,773 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:39,773 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-08 08:54:39,773 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:39,773 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:39,773 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:39,773 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. started... 2023-02-08 08:54:39,774 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 370d7626b49b9b3c93d59dd5a186a23e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:40,185 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103d6ce70633d714bb6b9b8ecb4844bc260 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103d6ce70633d714bb6b9b8ecb4844bc260 2023-02-08 08:54:40,591 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/aaa8daab32bd4624b149ee0b95b762f1 2023-02-08 08:54:40,596 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/aaa8daab32bd4624b149ee0b95b762f1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/aaa8daab32bd4624b149ee0b95b762f1 2023-02-08 08:54:40,600 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/aaa8daab32bd4624b149ee0b95b762f1, entries=1, sequenceid=33, filesize=5.0 K 2023-02-08 08:54:40,601 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 370d7626b49b9b3c93d59dd5a186a23e in 827ms, sequenceid=33, compaction requested=true 2023-02-08 08:54:40,601 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 370d7626b49b9b3c93d59dd5a186a23e: 2023-02-08 08:54:40,601 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:40,601 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:40,601 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:40,601 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:40,601 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-08 08:54:40,601 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-08 08:54:40,603 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:40,603 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:40,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:40,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:40,603 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,603 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:40,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:40,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:40,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:40,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:40,605 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-08 08:54:40,605 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@261bfd80[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:40,605 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-08 08:54:40,605 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,606 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,606 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-08 08:54:40,606 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-08 08:54:40,606 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:40,606 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:40,606 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:40,607 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-08 08:54:40,608 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:40,608 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:40,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:40,608 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-08 08:54:40,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:40,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:40,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:40,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:40,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:40,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:40,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:40,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:40,610 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-08 08:54:40,610 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-08 08:54:40,610 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:40,610 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:40,610 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,610 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:40,612 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,612 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:40,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:40,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,612 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:40,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:40,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:40,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:40,613 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:40,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:40,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:40,614 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:40,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:40,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:40,616 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:40,616 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,616 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:40,616 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:40,616 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:40,616 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:40,616 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:40,616 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 08:54:40,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:40,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:40,616 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,616 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,616 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:40,616 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:40,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:40,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:40,716 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:40,717 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:40,717 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:40,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:40,721 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:40,721 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:40,721 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:40,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:40,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,723 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 08:54:40,723 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:40,723 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-08 08:54:40,723 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-08 08:54:40,723 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,723 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,725 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:40,725 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:40,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:40,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:40,725 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:40,725 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:40,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,725 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:40,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,726 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,726 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-08 08:54:40,726 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:40,726 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-08 08:54:40,726 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:40,726 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-08 08:54:40,726 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:40,726 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:40,726 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:40,726 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. started... 2023-02-08 08:54:40,727 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 370d7626b49b9b3c93d59dd5a186a23e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:41,138 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509072faddf5257bd4c48b7a3a1c6c4d67cd8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201509072faddf5257bd4c48b7a3a1c6c4d67cd8 2023-02-08 08:54:41,543 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/ad5060a4a97242e7825a072a9095755d 2023-02-08 08:54:41,548 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/ad5060a4a97242e7825a072a9095755d as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/ad5060a4a97242e7825a072a9095755d 2023-02-08 08:54:41,551 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/ad5060a4a97242e7825a072a9095755d, entries=1, sequenceid=37, filesize=5.0 K 2023-02-08 08:54:41,552 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 370d7626b49b9b3c93d59dd5a186a23e in 826ms, sequenceid=37, compaction requested=true 2023-02-08 08:54:41,552 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 370d7626b49b9b3c93d59dd5a186a23e: 2023-02-08 08:54:41,552 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:41,552 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:41,552 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:41,553 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:41,553 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-08 08:54:41,553 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-08 08:54:41,554 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:41,554 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:41,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:41,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:41,554 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,554 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:41,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:41,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:41,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:41,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:41,555 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-08 08:54:41,555 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6debec92[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:41,556 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-08 08:54:41,556 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,557 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,558 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-08 08:54:41,558 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-08 08:54:41,558 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:41,558 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:41,558 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:41,559 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:41,559 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-08 08:54:41,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:41,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:41,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:41,559 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:41,559 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-08 08:54:41,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:41,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:41,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:41,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:41,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:41,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:41,561 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-08 08:54:41,561 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-08 08:54:41,561 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:41,561 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:41,561 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,561 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:41,562 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,562 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:41,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:41,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,562 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:41,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:41,563 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:41,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:41,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:41,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:41,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,564 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:41,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:41,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:41,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:41,570 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:41,570 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:41,570 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,570 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:41,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:41,571 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 08:54:41,571 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:41,571 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:41,571 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:41,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:41,571 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,571 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,571 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:41,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:41,571 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:41,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:41,671 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:41,671 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:41,672 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:41,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:41,675 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:41,675 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:41,676 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:41,677 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:41,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,678 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 08:54:41,678 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:41,678 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-08 08:54:41,678 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-08 08:54:41,678 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,678 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,679 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:41,679 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:41,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:41,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:41,679 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:41,680 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:41,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,680 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:41,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,680 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,681 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-08 08:54:41,681 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:41,681 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-08 08:54:41,681 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:41,681 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-08 08:54:41,681 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:41,681 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:41,681 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:41,681 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. started... 2023-02-08 08:54:41,681 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 370d7626b49b9b3c93d59dd5a186a23e 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:42,093 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112047c2595cd18e41fb9bacd8cd0b021b4b to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112047c2595cd18e41fb9bacd8cd0b021b4b 2023-02-08 08:54:42,498 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/f0b23f987b844bbcbc7a0494f021bcd8 2023-02-08 08:54:42,502 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/f0b23f987b844bbcbc7a0494f021bcd8 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/f0b23f987b844bbcbc7a0494f021bcd8 2023-02-08 08:54:42,506 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/f0b23f987b844bbcbc7a0494f021bcd8, entries=1, sequenceid=41, filesize=5.0 K 2023-02-08 08:54:42,507 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 370d7626b49b9b3c93d59dd5a186a23e in 826ms, sequenceid=41, compaction requested=true 2023-02-08 08:54:42,507 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 370d7626b49b9b3c93d59dd5a186a23e: 2023-02-08 08:54:42,507 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:42,507 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:42,507 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:42,507 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:42,507 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-08 08:54:42,507 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-08 08:54:42,509 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,509 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:42,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:42,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:42,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:42,509 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,509 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:42,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:42,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:42,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:42,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:42,510 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-08 08:54:42,510 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2492915b[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:42,510 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-08 08:54:42,510 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,511 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,512 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-08 08:54:42,512 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-08 08:54:42,512 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:42,512 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:42,512 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:42,514 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:42,514 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-08 08:54:42,514 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:42,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:42,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:42,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:42,514 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-08 08:54:42,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:42,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:42,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:42,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:42,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:42,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:42,516 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-08 08:54:42,516 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-08 08:54:42,516 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:42,516 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:42,516 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,516 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:42,517 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,517 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:42,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:42,517 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:42,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:42,518 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:42,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:42,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:42,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:42,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,519 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:42,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:42,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:42,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:42,521 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:42,521 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:42,521 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:42,521 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-08 08:54:42,521 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:42,521 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:42,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:42,521 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:42,521 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,521 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:42,521 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,521 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:42,521 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-08 08:54:42,521 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,521 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:42,521 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,521 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:42,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:42,621 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:42,621 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:42,621 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:42,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:42,623 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37869] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:42,625 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-08 08:54:42,625 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-08 08:54:42,626 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:54:42,626 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675847082626 2023-02-08 08:54:42,628 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 08:54:42,629 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-08 08:54:42,629 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 08:54:42,630 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] zookeeper.ReadOnlyZKClient(139): Connect 0x3f39dd85 to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:54:42,634 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3b5f8531, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:54:42,634 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:43,475 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103c59fb8e34df84dfba775a48129be4d61 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103c59fb8e34df84dfba775a48129be4d61 2023-02-08 08:54:43,476 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 08:54:43,488 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:54:43,489 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42154, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:54:43,495 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:43,495 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:43,495 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 08:54:43,497 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59366, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 08:54:43,506 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/5d537c7787fd4a04b95b1343cba760f2 first=Optional[r3] last=Optional[r6] 2023-02-08 08:54:43,506 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/5d537c7787fd4a04b95b1343cba760f2}] 2023-02-08 08:54:43,511 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/5d537c7787fd4a04b95b1343cba760f2 for inclusion in 370d7626b49b9b3c93d59dd5a186a23e/family1 2023-02-08 08:54:43,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-08 08:54:43,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 08:54:43,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for 370d7626b49b9b3c93d59dd5a186a23e: 2023-02-08 08:54:43,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/5d537c7787fd4a04b95b1343cba760f2 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kh4hdmjgris9jid7t4087s6e06sleveikvgcne2krfq8e9ugg47rp2ofmnk9amli/family1/5d537c7787fd4a04b95b1343cba760f2 2023-02-08 08:54:43,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kh4hdmjgris9jid7t4087s6e06sleveikvgcne2krfq8e9ugg47rp2ofmnk9amli/family1/5d537c7787fd4a04b95b1343cba760f2 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/86c86571a0c649219a8587f533269e3c_SeqId_44_ 2023-02-08 08:54:43,517 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kh4hdmjgris9jid7t4087s6e06sleveikvgcne2krfq8e9ugg47rp2ofmnk9amli/family1/5d537c7787fd4a04b95b1343cba760f2 into 370d7626b49b9b3c93d59dd5a186a23e/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/86c86571a0c649219a8587f533269e3c_SeqId_44_ - updating store file list. 2023-02-08 08:54:43,520 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/86c86571a0c649219a8587f533269e3c_SeqId_44_ into 370d7626b49b9b3c93d59dd5a186a23e/family1 2023-02-08 08:54:43,520 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kh4hdmjgris9jid7t4087s6e06sleveikvgcne2krfq8e9ugg47rp2ofmnk9amli/family1/5d537c7787fd4a04b95b1343cba760f2 into 370d7626b49b9b3c93d59dd5a186a23e/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/86c86571a0c649219a8587f533269e3c_SeqId_44_) 2023-02-08 08:54:43,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__kh4hdmjgris9jid7t4087s6e06sleveikvgcne2krfq8e9ugg47rp2ofmnk9amli/family1/5d537c7787fd4a04b95b1343cba760f2 2023-02-08 08:54:43,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-08 08:54:43,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-08 08:54:43,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(1912): 370d7626b49b9b3c93d59dd5a186a23e/family1 is initiating minor compaction 2023-02-08 08:54:43,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@39e6d9a8; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:43,525 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 370d7626b49b9b3c93d59dd5a186a23e/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:43,525 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/9616c3704c094ad8a8f79ad0669caed4, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/de16d57403b34f4bbb9740c85ffef7ab, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/861892cf416142ef8dbcef24da775976, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/5a229c63b0f144ccbdb29aab4ec9ae42, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/d029723c5c114d87b033577885af827b, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/ab83dcf35cb04506bf753f1141eaa9f2, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/aaa8daab32bd4624b149ee0b95b762f1, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/ad5060a4a97242e7825a072a9095755d, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/f0b23f987b844bbcbc7a0494f021bcd8, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/86c86571a0c649219a8587f533269e3c_SeqId_44_] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp, totalSize=50.4 K 2023-02-08 08:54:43,525 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 9616c3704c094ad8a8f79ad0669caed4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-08 08:54:43,525 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting de16d57403b34f4bbb9740c85ffef7ab, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-08 08:54:43,525 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 861892cf416142ef8dbcef24da775976, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-08 08:54:43,525 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 5a229c63b0f144ccbdb29aab4ec9ae42, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-08 08:54:43,525 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting d029723c5c114d87b033577885af827b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-08 08:54:43,525 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting ab83dcf35cb04506bf753f1141eaa9f2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-08 08:54:43,525 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting aaa8daab32bd4624b149ee0b95b762f1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-08 08:54:43,525 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting ad5060a4a97242e7825a072a9095755d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-08 08:54:43,525 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting f0b23f987b844bbcbc7a0494f021bcd8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-08 08:54:43,525 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 86c86571a0c649219a8587f533269e3c_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-08 08:54:43,526 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 08:54:43,527 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:43,529 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512283ba9d43649724e99b8003cff978c3f12 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512283ba9d43649724e99b8003cff978c3f12 2023-02-08 08:54:43,530 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231754e782c24b4492c974d5bcf56866c25 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231754e782c24b4492c974d5bcf56866c25 2023-02-08 08:54:43,531 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601019746410401e24123bf9a53573b408ca4 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601019746410401e24123bf9a53573b408ca4 2023-02-08 08:54:43,532 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103d6ce70633d714bb6b9b8ecb4844bc260 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103d6ce70633d714bb6b9b8ecb4844bc260 2023-02-08 08:54:43,532 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-02-08 08:54:43,532 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-08 08:54:43,532 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:43,532 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:43,532 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:43,532 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:43,533 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:43,533 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:43,533 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:43,533 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:43,550 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] throttle.PressureAwareThroughputController(145): 370d7626b49b9b3c93d59dd5a186a23e#family1#compaction#60 average throughput is 0.79 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-08 08:54:44,366 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/6433d6c9c91d4bf2b2510789277611bf as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/6433d6c9c91d4bf2b2510789277611bf 2023-02-08 08:54:44,371 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 370d7626b49b9b3c93d59dd5a186a23e/family1 of 370d7626b49b9b3c93d59dd5a186a23e into 6433d6c9c91d4bf2b2510789277611bf(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:44,371 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 370d7626b49b9b3c93d59dd5a186a23e: 2023-02-08 08:54:44,371 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1675846483524; duration=0sec 2023-02-08 08:54:44,371 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:44,375 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512056dd21d045198428d87d5fbd521262ce2 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512056dd21d045198428d87d5fbd521262ce2 2023-02-08 08:54:44,376 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 08:54:44,393 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:44,393 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:44,399 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1169e97882fd49119d65bcda7a323372 first=Optional[r00] last=Optional[r2] 2023-02-08 08:54:44,399 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1169e97882fd49119d65bcda7a323372}] 2023-02-08 08:54:44,401 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1169e97882fd49119d65bcda7a323372 for inclusion in 370d7626b49b9b3c93d59dd5a186a23e/family1 2023-02-08 08:54:44,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-08 08:54:44,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 08:54:44,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for 370d7626b49b9b3c93d59dd5a186a23e: 2023-02-08 08:54:44,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1169e97882fd49119d65bcda7a323372 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__q0qn7ju9a7r6i8ath3tfbj4ql65q3ag3aj804hrf96otvn02j75oef0cd0sni6i7/family1/1169e97882fd49119d65bcda7a323372 2023-02-08 08:54:44,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__q0qn7ju9a7r6i8ath3tfbj4ql65q3ag3aj804hrf96otvn02j75oef0cd0sni6i7/family1/1169e97882fd49119d65bcda7a323372 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/aea370d8b6b84542ad48cfc2f2502245_SeqId_47_ 2023-02-08 08:54:44,407 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__q0qn7ju9a7r6i8ath3tfbj4ql65q3ag3aj804hrf96otvn02j75oef0cd0sni6i7/family1/1169e97882fd49119d65bcda7a323372 into 370d7626b49b9b3c93d59dd5a186a23e/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/aea370d8b6b84542ad48cfc2f2502245_SeqId_47_ - updating store file list. 2023-02-08 08:54:44,410 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/aea370d8b6b84542ad48cfc2f2502245_SeqId_47_ into 370d7626b49b9b3c93d59dd5a186a23e/family1 2023-02-08 08:54:44,410 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__q0qn7ju9a7r6i8ath3tfbj4ql65q3ag3aj804hrf96otvn02j75oef0cd0sni6i7/family1/1169e97882fd49119d65bcda7a323372 into 370d7626b49b9b3c93d59dd5a186a23e/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/aea370d8b6b84542ad48cfc2f2502245_SeqId_47_) 2023-02-08 08:54:44,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__q0qn7ju9a7r6i8ath3tfbj4ql65q3ag3aj804hrf96otvn02j75oef0cd0sni6i7/family1/1169e97882fd49119d65bcda7a323372 2023-02-08 08:54:44,411 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-08 08:54:44,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-08 08:54:44,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(1912): 370d7626b49b9b3c93d59dd5a186a23e/family1 is initiating minor compaction (all files) 2023-02-08 08:54:44,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6c6a0fcd; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:44,412 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 370d7626b49b9b3c93d59dd5a186a23e/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:44,412 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/fa2a6f1e4e6d46529f9cdf442822ef0e, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/6433d6c9c91d4bf2b2510789277611bf, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/aea370d8b6b84542ad48cfc2f2502245_SeqId_47_] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp, totalSize=390.1 K 2023-02-08 08:54:44,413 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting fa2a6f1e4e6d46529f9cdf442822ef0e, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878836535 2023-02-08 08:54:44,413 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 6433d6c9c91d4bf2b2510789277611bf, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621236535 2023-02-08 08:54:44,413 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting aea370d8b6b84542ad48cfc2f2502245_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878836535 2023-02-08 08:54:44,414 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 08:54:44,414 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:44,416 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511303fad8d9c2ca841c88f271439b98f84e8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511303fad8d9c2ca841c88f271439b98f84e8 2023-02-08 08:54:44,417 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512014cb39f28925449cf88565065b766a235 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512014cb39f28925449cf88565065b766a235 2023-02-08 08:54:44,419 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120539b877a8167d4afb8da03e8c02d1c57f to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015120539b877a8167d4afb8da03e8c02d1c57f 2023-02-08 08:54:44,419 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-02-08 08:54:44,419 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-08 08:54:44,419 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:44,419 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 08:54:44,419 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:54:44,421 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-08 08:54:44,422 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.7970 sec 2023-02-08 08:54:44,434 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/9616c3704c094ad8a8f79ad0669caed4, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/de16d57403b34f4bbb9740c85ffef7ab, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/861892cf416142ef8dbcef24da775976, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/5a229c63b0f144ccbdb29aab4ec9ae42, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/d029723c5c114d87b033577885af827b, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/ab83dcf35cb04506bf753f1141eaa9f2, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/aaa8daab32bd4624b149ee0b95b762f1, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/ad5060a4a97242e7825a072a9095755d, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/f0b23f987b844bbcbc7a0494f021bcd8, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/86c86571a0c649219a8587f533269e3c_SeqId_44_] to archive 2023-02-08 08:54:44,434 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:44,436 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/9616c3704c094ad8a8f79ad0669caed4 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/9616c3704c094ad8a8f79ad0669caed4 2023-02-08 08:54:44,438 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/de16d57403b34f4bbb9740c85ffef7ab to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/de16d57403b34f4bbb9740c85ffef7ab 2023-02-08 08:54:44,439 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] throttle.PressureAwareThroughputController(145): 370d7626b49b9b3c93d59dd5a186a23e#family1#compaction#61 average throughput is 10.62 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-08 08:54:44,439 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/861892cf416142ef8dbcef24da775976 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/861892cf416142ef8dbcef24da775976 2023-02-08 08:54:44,441 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/5a229c63b0f144ccbdb29aab4ec9ae42 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/5a229c63b0f144ccbdb29aab4ec9ae42 2023-02-08 08:54:44,442 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/d029723c5c114d87b033577885af827b to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/d029723c5c114d87b033577885af827b 2023-02-08 08:54:44,444 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/ab83dcf35cb04506bf753f1141eaa9f2 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/ab83dcf35cb04506bf753f1141eaa9f2 2023-02-08 08:54:44,445 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/aaa8daab32bd4624b149ee0b95b762f1 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/aaa8daab32bd4624b149ee0b95b762f1 2023-02-08 08:54:44,446 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/ad5060a4a97242e7825a072a9095755d to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/ad5060a4a97242e7825a072a9095755d 2023-02-08 08:54:44,448 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/f0b23f987b844bbcbc7a0494f021bcd8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/f0b23f987b844bbcbc7a0494f021bcd8 2023-02-08 08:54:44,450 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/86c86571a0c649219a8587f533269e3c_SeqId_44_ to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/86c86571a0c649219a8587f533269e3c_SeqId_44_ 2023-02-08 08:54:44,507 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:44,507 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:44,508 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-08 08:54:44,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:44,861 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/.tmp/family1/16b4fb17774b425dab7a991b4e268a8a as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/16b4fb17774b425dab7a991b4e268a8a 2023-02-08 08:54:44,865 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 370d7626b49b9b3c93d59dd5a186a23e/family1 of 370d7626b49b9b3c93d59dd5a186a23e into 16b4fb17774b425dab7a991b4e268a8a(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:44,865 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 370d7626b49b9b3c93d59dd5a186a23e: 2023-02-08 08:54:44,865 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1675846484412; duration=0sec 2023-02-08 08:54:44,865 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:44,917 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-08 08:54:44,918 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-02-08 08:54:44,918 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-08 08:54:44,920 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-02-08 08:54:44,923 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=370d7626b49b9b3c93d59dd5a186a23e, REOPEN/MOVE}] 2023-02-08 08:54:44,923 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=370d7626b49b9b3c93d59dd5a186a23e, REOPEN/MOVE 2023-02-08 08:54:44,923 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/fa2a6f1e4e6d46529f9cdf442822ef0e, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/6433d6c9c91d4bf2b2510789277611bf, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/aea370d8b6b84542ad48cfc2f2502245_SeqId_47_] to archive 2023-02-08 08:54:44,924 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=370d7626b49b9b3c93d59dd5a186a23e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:44,924 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675846484924"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846484924"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846484924"}]},"ts":"1675846484924"} 2023-02-08 08:54:44,924 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:44,925 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 370d7626b49b9b3c93d59dd5a186a23e, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:54:44,926 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/fa2a6f1e4e6d46529f9cdf442822ef0e to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/fa2a6f1e4e6d46529f9cdf442822ef0e 2023-02-08 08:54:44,927 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/6433d6c9c91d4bf2b2510789277611bf to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/6433d6c9c91d4bf2b2510789277611bf 2023-02-08 08:54:44,928 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/aea370d8b6b84542ad48cfc2f2502245_SeqId_47_ to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/aea370d8b6b84542ad48cfc2f2502245_SeqId_47_ 2023-02-08 08:54:45,077 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:45,077 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 370d7626b49b9b3c93d59dd5a186a23e, disabling compactions & flushes 2023-02-08 08:54:45,077 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:45,077 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:45,077 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. after waiting 0 ms 2023-02-08 08:54:45,077 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:45,082 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-08 08:54:45,083 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:45,083 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 370d7626b49b9b3c93d59dd5a186a23e: Waiting for close lock at 1675846485077Running coprocessor pre-close hooks at 1675846485077Disabling compacts and flushes for region at 1675846485077Disabling writes for close at 1675846485077Writing region close event to WAL at 1675846485079 (+2 ms)Running coprocessor post-close hooks at 1675846485083 (+4 ms)Closed at 1675846485083 2023-02-08 08:54:45,083 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 370d7626b49b9b3c93d59dd5a186a23e to self. 2023-02-08 08:54:45,084 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:45,085 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=370d7626b49b9b3c93d59dd5a186a23e, regionState=CLOSED 2023-02-08 08:54:45,085 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675846485085"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846485085"}]},"ts":"1675846485085"} 2023-02-08 08:54:45,087 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-08 08:54:45,087 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 370d7626b49b9b3c93d59dd5a186a23e, server=jenkins-hbase4.apache.org,42239,1675846413348 in 161 msec 2023-02-08 08:54:45,088 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=370d7626b49b9b3c93d59dd5a186a23e, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=true 2023-02-08 08:54:45,238 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=370d7626b49b9b3c93d59dd5a186a23e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:45,238 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675846485238"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846485238"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846485238"}]},"ts":"1675846485238"} 2023-02-08 08:54:45,240 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure 370d7626b49b9b3c93d59dd5a186a23e, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:54:45,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:45,394 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 370d7626b49b9b3c93d59dd5a186a23e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:54:45,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:45,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:45,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:45,395 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:45,396 INFO [StoreOpener-370d7626b49b9b3c93d59dd5a186a23e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:45,397 DEBUG [StoreOpener-370d7626b49b9b3c93d59dd5a186a23e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1 2023-02-08 08:54:45,397 DEBUG [StoreOpener-370d7626b49b9b3c93d59dd5a186a23e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1 2023-02-08 08:54:45,397 INFO [StoreOpener-370d7626b49b9b3c93d59dd5a186a23e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 370d7626b49b9b3c93d59dd5a186a23e columnFamilyName family1 2023-02-08 08:54:45,401 DEBUG [StoreOpener-370d7626b49b9b3c93d59dd5a186a23e-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/16b4fb17774b425dab7a991b4e268a8a 2023-02-08 08:54:45,401 INFO [StoreOpener-370d7626b49b9b3c93d59dd5a186a23e-1] regionserver.HStore(310): Store=370d7626b49b9b3c93d59dd5a186a23e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:45,402 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:45,403 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:45,405 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:45,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 370d7626b49b9b3c93d59dd5a186a23e; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67941912, jitterRate=0.012413382530212402}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 08:54:45,406 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 370d7626b49b9b3c93d59dd5a186a23e: Running coprocessor pre-open hook at 1675846485395Writing region info on filesystem at 1675846485395Initializing all the Stores at 1675846485395Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675846485395Cleaning up temporary data from old regions at 1675846485403 (+8 ms)Cleaning up detritus from prior splits at 1675846485404 (+1 ms)Running coprocessor post-open hooks at 1675846485406 (+2 ms)Region opened successfully at 1675846485406 2023-02-08 08:54:45,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e., pid=47, masterSystemTime=1675846485391 2023-02-08 08:54:45,408 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:45,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:45,408 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=370d7626b49b9b3c93d59dd5a186a23e, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:45,408 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675846485408"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846485408"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846485408"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846485408"}]},"ts":"1675846485408"} 2023-02-08 08:54:45,411 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-02-08 08:54:45,411 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure 370d7626b49b9b3c93d59dd5a186a23e, server=jenkins-hbase4.apache.org,42239,1675846413348 in 170 msec 2023-02-08 08:54:45,412 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-08 08:54:45,412 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=370d7626b49b9b3c93d59dd5a186a23e, REOPEN/MOVE in 488 msec 2023-02-08 08:54:45,414 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-02-08 08:54:45,414 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 493 msec 2023-02-08 08:54:45,415 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 907 msec 2023-02-08 08:54:45,416 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:45,416 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:45,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-02-08 08:54:45,416 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:45,416 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:45,417 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:45,417 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:45,417 INFO [Listener at localhost/40763] compactions.TestMobCompactor(278): alter status finished 2023-02-08 08:54:45,418 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37869] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:45,419 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-08 08:54:45,420 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-08 08:54:45,420 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675847085420 2023-02-08 08:54:45,420 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:54:45,421 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 08:54:45,422 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-08 08:54:45,422 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 08:54:45,422 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] zookeeper.ReadOnlyZKClient(139): Connect 0x2736c997 to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:54:45,426 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6007b64d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:54:45,426 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:45,427 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:45,427 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:45,427 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:45,427 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:46,256 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112872a5b8a351b64c32b5c2b687bb2369c9 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112872a5b8a351b64c32b5c2b687bb2369c9 2023-02-08 08:54:46,257 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 08:54:46,269 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:54:46,270 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42166, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:54:46,275 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:46,275 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:46,276 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 08:54:46,277 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59378, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 08:54:46,285 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/e4707d8eee844472b99cdef28ed06ff7 first=Optional[r05] last=Optional[r8] 2023-02-08 08:54:46,286 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=51 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/e4707d8eee844472b99cdef28ed06ff7}] 2023-02-08 08:54:46,290 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/e4707d8eee844472b99cdef28ed06ff7 for inclusion in 370d7626b49b9b3c93d59dd5a186a23e/family1 2023-02-08 08:54:46,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-08 08:54:46,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 08:54:46,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for 370d7626b49b9b3c93d59dd5a186a23e: 2023-02-08 08:54:46,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/e4707d8eee844472b99cdef28ed06ff7 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__pkinj7n18m1mrc514kn53vebj2qbiku61644v5734189f9gnfrrn9gab6qb08i2l/family1/e4707d8eee844472b99cdef28ed06ff7 2023-02-08 08:54:46,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__pkinj7n18m1mrc514kn53vebj2qbiku61644v5734189f9gnfrrn9gab6qb08i2l/family1/e4707d8eee844472b99cdef28ed06ff7 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/18cdc9736a03466b815b0c85c808e2e4_SeqId_53_ 2023-02-08 08:54:46,296 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__pkinj7n18m1mrc514kn53vebj2qbiku61644v5734189f9gnfrrn9gab6qb08i2l/family1/e4707d8eee844472b99cdef28ed06ff7 into 370d7626b49b9b3c93d59dd5a186a23e/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/18cdc9736a03466b815b0c85c808e2e4_SeqId_53_ - updating store file list. 2023-02-08 08:54:46,299 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/18cdc9736a03466b815b0c85c808e2e4_SeqId_53_ into 370d7626b49b9b3c93d59dd5a186a23e/family1 2023-02-08 08:54:46,299 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__pkinj7n18m1mrc514kn53vebj2qbiku61644v5734189f9gnfrrn9gab6qb08i2l/family1/e4707d8eee844472b99cdef28ed06ff7 into 370d7626b49b9b3c93d59dd5a186a23e/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/18cdc9736a03466b815b0c85c808e2e4_SeqId_53_) 2023-02-08 08:54:46,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__pkinj7n18m1mrc514kn53vebj2qbiku61644v5734189f9gnfrrn9gab6qb08i2l/family1/e4707d8eee844472b99cdef28ed06ff7 2023-02-08 08:54:46,301 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 08:54:46,302 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:46,304 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112047c2595cd18e41fb9bacd8cd0b021b4b to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112047c2595cd18e41fb9bacd8cd0b021b4b 2023-02-08 08:54:46,305 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112881b190420da8413aae0e777bdfbfa706 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015112881b190420da8413aae0e777bdfbfa706 2023-02-08 08:54:46,305 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-02-08 08:54:46,305 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-08 08:54:46,305 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:46,305 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:46,305 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:46,305 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 08:54:46,306 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:54:46,307 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-08 08:54:46,308 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 888 msec 2023-02-08 08:54:46,356 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:46,356 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:46,357 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-08 08:54:46,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:46,766 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-02-08 08:54:46,767 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-08 08:54:46,767 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-02-08 08:54:46,769 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-02-08 08:54:46,772 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=370d7626b49b9b3c93d59dd5a186a23e, REOPEN/MOVE}] 2023-02-08 08:54:46,772 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=370d7626b49b9b3c93d59dd5a186a23e, REOPEN/MOVE 2023-02-08 08:54:46,773 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=370d7626b49b9b3c93d59dd5a186a23e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:46,773 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675846486773"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846486773"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846486773"}]},"ts":"1675846486773"} 2023-02-08 08:54:46,774 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure 370d7626b49b9b3c93d59dd5a186a23e, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:54:46,926 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:46,926 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 370d7626b49b9b3c93d59dd5a186a23e, disabling compactions & flushes 2023-02-08 08:54:46,926 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:46,926 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:46,926 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. after waiting 0 ms 2023-02-08 08:54:46,926 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:46,932 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-08 08:54:46,933 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:46,933 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 370d7626b49b9b3c93d59dd5a186a23e: Waiting for close lock at 1675846486926Running coprocessor pre-close hooks at 1675846486926Disabling compacts and flushes for region at 1675846486926Disabling writes for close at 1675846486926Writing region close event to WAL at 1675846486929 (+3 ms)Running coprocessor post-close hooks at 1675846486933 (+4 ms)Closed at 1675846486933 2023-02-08 08:54:46,933 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 370d7626b49b9b3c93d59dd5a186a23e to self. 2023-02-08 08:54:46,935 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:46,935 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=370d7626b49b9b3c93d59dd5a186a23e, regionState=CLOSED 2023-02-08 08:54:46,935 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675846486935"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846486935"}]},"ts":"1675846486935"} 2023-02-08 08:54:46,937 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-02-08 08:54:46,937 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure 370d7626b49b9b3c93d59dd5a186a23e, server=jenkins-hbase4.apache.org,42239,1675846413348 in 162 msec 2023-02-08 08:54:46,938 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=370d7626b49b9b3c93d59dd5a186a23e, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=true 2023-02-08 08:54:47,088 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=370d7626b49b9b3c93d59dd5a186a23e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:47,088 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675846487088"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846487088"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846487088"}]},"ts":"1675846487088"} 2023-02-08 08:54:47,090 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure 370d7626b49b9b3c93d59dd5a186a23e, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:54:47,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:47,244 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 370d7626b49b9b3c93d59dd5a186a23e, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:54:47,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:47,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:47,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:47,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:47,246 INFO [StoreOpener-370d7626b49b9b3c93d59dd5a186a23e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:47,247 DEBUG [StoreOpener-370d7626b49b9b3c93d59dd5a186a23e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1 2023-02-08 08:54:47,247 DEBUG [StoreOpener-370d7626b49b9b3c93d59dd5a186a23e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1 2023-02-08 08:54:47,247 INFO [StoreOpener-370d7626b49b9b3c93d59dd5a186a23e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 370d7626b49b9b3c93d59dd5a186a23e columnFamilyName family1 2023-02-08 08:54:47,252 DEBUG [StoreOpener-370d7626b49b9b3c93d59dd5a186a23e-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/16b4fb17774b425dab7a991b4e268a8a 2023-02-08 08:54:47,255 DEBUG [StoreOpener-370d7626b49b9b3c93d59dd5a186a23e-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/family1/18cdc9736a03466b815b0c85c808e2e4_SeqId_53_ 2023-02-08 08:54:47,255 INFO [StoreOpener-370d7626b49b9b3c93d59dd5a186a23e-1] regionserver.HStore(310): Store=370d7626b49b9b3c93d59dd5a186a23e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:47,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:47,257 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:47,260 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:54:47,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 370d7626b49b9b3c93d59dd5a186a23e; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64712291, jitterRate=-0.03571172058582306}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 08:54:47,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 370d7626b49b9b3c93d59dd5a186a23e: Running coprocessor pre-open hook at 1675846487245Writing region info on filesystem at 1675846487245Initializing all the Stores at 1675846487246 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675846487246Cleaning up temporary data from old regions at 1675846487258 (+12 ms)Cleaning up detritus from prior splits at 1675846487258Running coprocessor post-open hooks at 1675846487261 (+3 ms)Region opened successfully at 1675846487261 2023-02-08 08:54:47,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e., pid=53, masterSystemTime=1675846487241 2023-02-08 08:54:47,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:47,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:54:47,263 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=370d7626b49b9b3c93d59dd5a186a23e, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:47,263 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1675846487263"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846487263"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846487263"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846487263"}]},"ts":"1675846487263"} 2023-02-08 08:54:47,266 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-02-08 08:54:47,266 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure 370d7626b49b9b3c93d59dd5a186a23e, server=jenkins-hbase4.apache.org,42239,1675846413348 in 174 msec 2023-02-08 08:54:47,267 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-08 08:54:47,267 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=370d7626b49b9b3c93d59dd5a186a23e, REOPEN/MOVE in 495 msec 2023-02-08 08:54:47,269 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-08 08:54:47,269 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 499 msec 2023-02-08 08:54:47,271 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 913 msec 2023-02-08 08:54:47,271 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:47,271 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:47,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-08 08:54:47,272 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:47,272 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:47,273 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:47,273 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:47,273 INFO [Listener at localhost/40763] compactions.TestMobCompactor(278): alter status finished 2023-02-08 08:54:47,274 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37869] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-08 08:54:47,275 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-08 08:54:47,276 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-08 08:54:47,276 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:54:47,276 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675847087276 2023-02-08 08:54:47,277 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 08:54:47,277 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-08 08:54:47,278 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 08:54:47,278 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] zookeeper.ReadOnlyZKClient(139): Connect 0x38487ead to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:54:47,282 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@65a00237, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:54:47,282 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:47,282 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:47,282 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:47,282 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:47,282 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:47,282 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:47,282 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:47,282 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-08 08:54:47,282 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-08 08:54:47,282 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:54:47,284 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-08 08:54:47,285 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 9 msec 2023-02-08 08:54:47,331 INFO [Listener at localhost/40763] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=242 (was 263), OpenFileDescriptor=606 (was 629), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=270 (was 328), ProcessCount=168 (was 168), AvailableMemoryMB=4185 (was 4225) 2023-02-08 08:54:47,341 INFO [Listener at localhost/40763] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=242, OpenFileDescriptor=606, MaxFileDescriptor=60000, SystemLoadAverage=270, ProcessCount=168, AvailableMemoryMB=4184 2023-02-08 08:54:47,342 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:47,342 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:47,343 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-08 08:54:47,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:47,345 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:54:47,345 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-02-08 08:54:47,345 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:47,346 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:47,346 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:54:47,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-08 08:54:47,347 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:54:47,347 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900 empty. 2023-02-08 08:54:47,348 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:54:47,348 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-02-08 08:54:47,446 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:47,447 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:47,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-08 08:54:47,647 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:47,648 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:47,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-08 08:54:47,755 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-02-08 08:54:47,756 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7671): creating {ENCODED => 201d231772cef3ab7c7efad6d0fb3900, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp 2023-02-08 08:54:47,949 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:47,949 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:47,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-08 08:54:48,162 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:48,162 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1603): Closing 201d231772cef3ab7c7efad6d0fb3900, disabling compactions & flushes 2023-02-08 08:54:48,162 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:48,162 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:48,162 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. after waiting 0 ms 2023-02-08 08:54:48,162 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:48,162 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:48,162 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1557): Region close journal for 201d231772cef3ab7c7efad6d0fb3900: Waiting for close lock at 1675846488162Disabling compacts and flushes for region at 1675846488162Disabling writes for close at 1675846488162Writing region close event to WAL at 1675846488162Closed at 1675846488162 2023-02-08 08:54:48,164 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:54:48,165 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675846488165"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846488165"}]},"ts":"1675846488165"} 2023-02-08 08:54:48,166 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 08:54:48,167 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:54:48,167 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846488167"}]},"ts":"1675846488167"} 2023-02-08 08:54:48,168 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-02-08 08:54:48,171 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=201d231772cef3ab7c7efad6d0fb3900, ASSIGN}] 2023-02-08 08:54:48,173 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=201d231772cef3ab7c7efad6d0fb3900, ASSIGN 2023-02-08 08:54:48,173 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=201d231772cef3ab7c7efad6d0fb3900, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=false 2023-02-08 08:54:48,324 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=201d231772cef3ab7c7efad6d0fb3900, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:48,324 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675846488324"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846488324"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846488324"}]},"ts":"1675846488324"} 2023-02-08 08:54:48,326 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure 201d231772cef3ab7c7efad6d0fb3900, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:54:48,450 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:48,450 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:48,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-08 08:54:48,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:48,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 201d231772cef3ab7c7efad6d0fb3900, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:54:48,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:54:48,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:54:48,481 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:54:48,481 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:54:48,482 INFO [StoreOpener-201d231772cef3ab7c7efad6d0fb3900-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:54:48,483 DEBUG [StoreOpener-201d231772cef3ab7c7efad6d0fb3900-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1 2023-02-08 08:54:48,483 DEBUG [StoreOpener-201d231772cef3ab7c7efad6d0fb3900-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1 2023-02-08 08:54:48,483 INFO [StoreOpener-201d231772cef3ab7c7efad6d0fb3900-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 201d231772cef3ab7c7efad6d0fb3900 columnFamilyName family1 2023-02-08 08:54:48,485 INFO [StoreOpener-201d231772cef3ab7c7efad6d0fb3900-1] regionserver.HStore(310): Store=201d231772cef3ab7c7efad6d0fb3900/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:54:48,485 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:54:48,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:54:48,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:54:48,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:54:48,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 201d231772cef3ab7c7efad6d0fb3900; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73671590, jitterRate=0.09779223799705505}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 08:54:48,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 201d231772cef3ab7c7efad6d0fb3900: Running coprocessor pre-open hook at 1675846488481Writing region info on filesystem at 1675846488481Initializing all the Stores at 1675846488481Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1675846488481Cleaning up temporary data from old regions at 1675846488486 (+5 ms)Cleaning up detritus from prior splits at 1675846488487 (+1 ms)Running coprocessor post-open hooks at 1675846488491 (+4 ms)Region opened successfully at 1675846488491 2023-02-08 08:54:48,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900., pid=57, masterSystemTime=1675846488477 2023-02-08 08:54:48,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:48,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:48,494 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=201d231772cef3ab7c7efad6d0fb3900, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:48,494 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675846488494"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846488494"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846488494"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846488494"}]},"ts":"1675846488494"} 2023-02-08 08:54:48,496 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-08 08:54:48,496 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure 201d231772cef3ab7c7efad6d0fb3900, server=jenkins-hbase4.apache.org,42239,1675846413348 in 169 msec 2023-02-08 08:54:48,498 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-02-08 08:54:48,498 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=201d231772cef3ab7c7efad6d0fb3900, ASSIGN in 325 msec 2023-02-08 08:54:48,498 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:54:48,498 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846488498"}]},"ts":"1675846488498"} 2023-02-08 08:54:48,499 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-02-08 08:54:48,501 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:54:48,502 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1580 sec 2023-02-08 08:54:49,451 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:49,451 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:49,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-08 08:54:49,452 INFO [Listener at localhost/40763] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-02-08 08:54:49,470 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:49,470 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:49,471 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:49,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:49,473 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 08:54:49,473 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:49,473 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-08 08:54:49,473 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-08 08:54:49,474 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:49,474 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:49,475 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:49,475 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:49,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:49,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:49,475 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:49,475 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:49,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:49,476 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:49,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:49,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:49,476 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:49,476 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:49,476 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-08 08:54:49,476 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:49,476 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-08 08:54:49,478 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:49,478 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:49,478 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:49,478 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. started... 2023-02-08 08:54:49,479 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 201d231772cef3ab7c7efad6d0fb3900 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-08 08:54:49,900 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130149cab94b7d74f69a099caec265c49a4 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130149cab94b7d74f69a099caec265c49a4 2023-02-08 08:54:50,305 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/257e1b0da8504edda13d06572114663a 2023-02-08 08:54:50,309 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/257e1b0da8504edda13d06572114663a as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/257e1b0da8504edda13d06572114663a 2023-02-08 08:54:50,313 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/257e1b0da8504edda13d06572114663a, entries=1000, sequenceid=5, filesize=190.5 K 2023-02-08 08:54:50,313 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 201d231772cef3ab7c7efad6d0fb3900 in 834ms, sequenceid=5, compaction requested=false 2023-02-08 08:54:50,314 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 08:54:50,314 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 201d231772cef3ab7c7efad6d0fb3900: 2023-02-08 08:54:50,314 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:50,314 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:50,314 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:50,314 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:50,314 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-08 08:54:50,314 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-08 08:54:50,316 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,316 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:50,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:50,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:50,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:50,316 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,316 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:50,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:50,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:50,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:50,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:50,317 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-08 08:54:50,317 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47981a44[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:50,317 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-08 08:54:50,317 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,318 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,318 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-08 08:54:50,318 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-08 08:54:50,319 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:50,319 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:50,319 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:50,320 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-08 08:54:50,320 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:50,320 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:50,320 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-08 08:54:50,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:50,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:50,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:50,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:50,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:50,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:50,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:50,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:50,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:50,322 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-08 08:54:50,322 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-08 08:54:50,322 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:50,322 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:50,322 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,322 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:50,324 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,324 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,324 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:50,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:50,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:50,324 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:50,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:50,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:50,325 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:50,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:50,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:50,325 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:50,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:50,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:50,330 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:50,330 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:50,330 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,330 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:50,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 08:54:50,331 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:50,330 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,330 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:50,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:50,330 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:50,331 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:50,331 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,331 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:50,331 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,331 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:50,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:50,331 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,331 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:50,431 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:50,431 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:50,431 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:50,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:50,435 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:50,435 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:50,436 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:50,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:50,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,437 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 08:54:50,437 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:50,437 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-08 08:54:50,437 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-08 08:54:50,438 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,438 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,439 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:50,439 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:50,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:50,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:50,439 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:50,439 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:50,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,439 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:50,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,440 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,440 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-08 08:54:50,440 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:50,440 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-08 08:54:50,440 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:50,440 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-08 08:54:50,440 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:50,440 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:50,440 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:50,440 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. started... 2023-02-08 08:54:50,440 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 201d231772cef3ab7c7efad6d0fb3900 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-08 08:54:50,610 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 08:54:50,852 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511284cee06780f5241e6af2751e6070e2b9c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511284cee06780f5241e6af2751e6070e2b9c 2023-02-08 08:54:51,257 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/72bbd8868af749a2a331b94d040012cc 2023-02-08 08:54:51,261 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/72bbd8868af749a2a331b94d040012cc as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/72bbd8868af749a2a331b94d040012cc 2023-02-08 08:54:51,265 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/72bbd8868af749a2a331b94d040012cc, entries=1, sequenceid=9, filesize=5.0 K 2023-02-08 08:54:51,266 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 201d231772cef3ab7c7efad6d0fb3900 in 826ms, sequenceid=9, compaction requested=false 2023-02-08 08:54:51,266 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 201d231772cef3ab7c7efad6d0fb3900: 2023-02-08 08:54:51,266 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:51,266 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:51,266 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:51,266 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:51,266 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-08 08:54:51,266 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-08 08:54:51,269 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:51,269 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:51,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:51,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:51,269 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,269 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:51,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:51,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:51,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:51,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:51,270 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-08 08:54:51,270 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e9618ca[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:51,270 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-08 08:54:51,271 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,272 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,272 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-08 08:54:51,272 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-08 08:54:51,272 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:51,272 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:51,272 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:51,273 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-08 08:54:51,273 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:51,273 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:51,273 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-08 08:54:51,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:51,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:51,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:51,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:51,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:51,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:51,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:51,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:51,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:51,275 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-08 08:54:51,275 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-08 08:54:51,275 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:51,275 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:51,275 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,275 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:51,277 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,277 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:51,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:51,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,277 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:51,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:51,278 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:51,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:51,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:51,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:51,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,279 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:51,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:51,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:51,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:51,281 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:51,281 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,281 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,281 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:51,281 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,281 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,281 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 08:54:51,281 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:51,281 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:51,281 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:51,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:51,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:51,281 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:51,281 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,282 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-08 08:54:51,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:51,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:51,282 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:51,282 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:51,382 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:51,382 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:51,382 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:51,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:51,386 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:51,386 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:51,387 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:51,389 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:51,389 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,389 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 08:54:51,389 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:51,389 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-08 08:54:51,389 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-08 08:54:51,389 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,389 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,390 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:51,390 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:51,390 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:51,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:51,391 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:51,391 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:51,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,391 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:51,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,391 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,392 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-08 08:54:51,392 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:51,392 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-08 08:54:51,392 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:51,392 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-08 08:54:51,392 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:51,392 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:51,392 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:51,392 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. started... 2023-02-08 08:54:51,392 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 201d231772cef3ab7c7efad6d0fb3900 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:51,804 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512015ff1e733d1ac437dafbbeff0e2c0723c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512015ff1e733d1ac437dafbbeff0e2c0723c 2023-02-08 08:54:52,209 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/78aef3deed474e50a30b1fd7ce9a2ae9 2023-02-08 08:54:52,213 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/78aef3deed474e50a30b1fd7ce9a2ae9 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/78aef3deed474e50a30b1fd7ce9a2ae9 2023-02-08 08:54:52,217 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/78aef3deed474e50a30b1fd7ce9a2ae9, entries=1, sequenceid=13, filesize=5.0 K 2023-02-08 08:54:52,218 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 201d231772cef3ab7c7efad6d0fb3900 in 826ms, sequenceid=13, compaction requested=true 2023-02-08 08:54:52,218 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 201d231772cef3ab7c7efad6d0fb3900: 2023-02-08 08:54:52,218 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:52,218 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:52,218 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:52,218 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:52,218 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-08 08:54:52,218 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-08 08:54:52,219 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,219 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:52,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:52,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:52,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:52,220 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,220 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:52,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:52,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:52,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:52,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:52,221 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-08 08:54:52,221 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@cabfcf6[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:52,221 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-08 08:54:52,221 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,222 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,222 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,222 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-08 08:54:52,222 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-08 08:54:52,222 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:52,222 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:52,222 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:52,229 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:52,229 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-08 08:54:52,229 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:52,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:52,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:52,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:52,229 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-08 08:54:52,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:52,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:52,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:52,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:52,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:52,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:52,231 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-08 08:54:52,231 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-08 08:54:52,231 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:52,231 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:52,231 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,231 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:52,232 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,232 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:52,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:52,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,232 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:52,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:52,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:52,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:52,233 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:52,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:52,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,234 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:52,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:52,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:52,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:52,236 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:52,236 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:52,236 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:52,236 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 08:54:52,236 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:52,236 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-08 08:54:52,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:52,236 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:52,236 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,236 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:52,236 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:52,236 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,236 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:52,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:52,236 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:52,236 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:52,236 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,336 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:52,337 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:52,337 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:52,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:52,340 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:52,340 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:52,341 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:52,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:52,342 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,342 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 08:54:52,342 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:52,343 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-08 08:54:52,343 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-08 08:54:52,343 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,343 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,344 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:52,344 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:52,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:52,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:52,344 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:52,344 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:52,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,345 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:52,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,345 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,345 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-08 08:54:52,345 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:52,345 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-08 08:54:52,345 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:52,345 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-08 08:54:52,345 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:52,345 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:52,345 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:52,346 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. started... 2023-02-08 08:54:52,346 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 201d231772cef3ab7c7efad6d0fb3900 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:52,758 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205a68e59eaa2f049faaf1b95010df1912c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205a68e59eaa2f049faaf1b95010df1912c 2023-02-08 08:54:53,163 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/32cc67c20da34e3999f16ed8b12d0507 2023-02-08 08:54:53,170 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/32cc67c20da34e3999f16ed8b12d0507 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/32cc67c20da34e3999f16ed8b12d0507 2023-02-08 08:54:53,175 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/32cc67c20da34e3999f16ed8b12d0507, entries=1, sequenceid=17, filesize=5.0 K 2023-02-08 08:54:53,175 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 201d231772cef3ab7c7efad6d0fb3900 in 829ms, sequenceid=17, compaction requested=true 2023-02-08 08:54:53,175 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 201d231772cef3ab7c7efad6d0fb3900: 2023-02-08 08:54:53,175 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:53,175 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:53,176 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:53,176 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:53,176 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-08 08:54:53,176 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-08 08:54:53,179 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:53,179 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:53,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:53,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:53,179 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,179 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:53,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:53,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:53,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:53,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:53,180 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-08 08:54:53,181 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3c83ac71[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:53,181 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-08 08:54:53,181 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,184 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,185 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-08 08:54:53,185 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-08 08:54:53,185 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:53,185 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:53,185 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:53,186 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-08 08:54:53,186 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:53,186 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:53,186 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-08 08:54:53,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:53,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:53,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:53,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:53,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:53,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:53,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:53,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:53,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:53,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-08 08:54:53,188 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-08 08:54:53,188 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:53,188 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:53,188 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,189 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:53,191 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,191 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,191 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:53,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:53,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:53,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:53,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:53,192 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:53,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:53,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:53,193 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:53,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:53,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:53,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,195 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:53,195 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:53,195 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:53,195 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:53,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:53,195 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 08:54:53,195 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:53,195 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,195 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-08 08:54:53,195 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:53,195 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:53,195 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:53,195 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,196 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:53,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:53,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:53,196 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,196 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,296 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:53,296 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:53,296 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:53,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:53,300 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:53,300 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:53,300 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:53,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:53,302 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,302 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 08:54:53,302 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:53,302 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-08 08:54:53,302 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-08 08:54:53,302 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,302 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,304 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:53,304 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:53,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:53,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:53,304 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:53,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,304 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:53,304 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:53,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,305 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,305 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-08 08:54:53,305 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:53,305 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-08 08:54:53,305 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:53,305 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-08 08:54:53,305 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:53,305 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:53,305 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:53,305 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. started... 2023-02-08 08:54:53,305 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 201d231772cef3ab7c7efad6d0fb3900 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:53,717 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512283a08d51fed9541569b3436885aaed6a8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512283a08d51fed9541569b3436885aaed6a8 2023-02-08 08:54:54,122 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/a59d939ea0c24c11b09c8550aaecc175 2023-02-08 08:54:54,127 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/a59d939ea0c24c11b09c8550aaecc175 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/a59d939ea0c24c11b09c8550aaecc175 2023-02-08 08:54:54,130 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/a59d939ea0c24c11b09c8550aaecc175, entries=1, sequenceid=21, filesize=5.0 K 2023-02-08 08:54:54,131 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 201d231772cef3ab7c7efad6d0fb3900 in 826ms, sequenceid=21, compaction requested=true 2023-02-08 08:54:54,131 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 201d231772cef3ab7c7efad6d0fb3900: 2023-02-08 08:54:54,131 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:54,131 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:54,131 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:54,131 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:54,131 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-08 08:54:54,131 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-08 08:54:54,132 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,132 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:54,133 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,133 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:54,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:54,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:54,134 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-08 08:54:54,134 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b4edb9f[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:54,134 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-08 08:54:54,134 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,137 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,137 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-08 08:54:54,137 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-08 08:54:54,137 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:54,137 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:54,137 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:54,138 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-08 08:54:54,138 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:54,138 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:54,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:54,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:54,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:54,138 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-08 08:54:54,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:54,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:54,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:54,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-08 08:54:54,140 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-08 08:54:54,140 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:54,140 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:54,140 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,140 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:54,142 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,142 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,142 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:54,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:54,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:54,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:54,142 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:54,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:54,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:54,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,143 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:54,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:54,145 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:54,145 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,145 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:54,145 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:54,145 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-08 08:54:54,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:54,145 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 08:54:54,145 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:54,145 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:54,145 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:54,145 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:54,146 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:54,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:54,146 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,146 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:54,146 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:54,146 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,246 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:54,246 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:54,246 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:54,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:54,250 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:54,250 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:54,250 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:54,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:54,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,252 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 08:54:54,252 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:54,252 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-08 08:54:54,252 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-08 08:54:54,252 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,253 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,254 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:54,255 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:54,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:54,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:54,255 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:54,255 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:54,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,255 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:54,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,256 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,256 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-08 08:54:54,256 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:54,256 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-08 08:54:54,256 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:54,256 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-08 08:54:54,256 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:54,256 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:54,256 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:54,256 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. started... 2023-02-08 08:54:54,256 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 201d231772cef3ab7c7efad6d0fb3900 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:54,667 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231c45f8bf6fd404daebd7bd8042e8f102c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231c45f8bf6fd404daebd7bd8042e8f102c 2023-02-08 08:54:55,072 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/5eb1bd4950a644788e26c4aa4d53b3cf 2023-02-08 08:54:55,077 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/5eb1bd4950a644788e26c4aa4d53b3cf as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/5eb1bd4950a644788e26c4aa4d53b3cf 2023-02-08 08:54:55,080 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/5eb1bd4950a644788e26c4aa4d53b3cf, entries=1, sequenceid=25, filesize=5.0 K 2023-02-08 08:54:55,081 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 201d231772cef3ab7c7efad6d0fb3900 in 825ms, sequenceid=25, compaction requested=true 2023-02-08 08:54:55,081 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 201d231772cef3ab7c7efad6d0fb3900: 2023-02-08 08:54:55,081 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:55,081 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:55,081 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:55,081 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:55,081 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-08 08:54:55,081 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-08 08:54:55,082 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,082 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:55,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:55,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:55,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:55,083 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,083 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:55,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:55,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:55,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:55,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:55,084 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-08 08:54:55,084 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3510d474[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:55,084 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-08 08:54:55,084 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,085 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,085 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-08 08:54:55,085 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:55,085 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:55,085 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-08 08:54:55,085 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:55,087 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-08 08:54:55,087 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:55,087 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:55,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:55,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:55,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:55,087 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-08 08:54:55,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:55,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:55,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:55,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:55,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:55,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:55,089 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-08 08:54:55,089 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-08 08:54:55,089 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:55,089 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:55,089 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,089 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:55,091 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,091 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:55,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:55,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,091 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:55,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:55,091 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:55,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:55,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:55,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:55,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,092 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:55,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:55,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:55,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:55,096 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:55,097 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,097 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:55,097 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:55,097 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 08:54:55,097 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:55,097 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:55,097 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:55,097 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:55,097 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-08 08:54:55,097 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:55,097 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:55,097 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:55,097 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,097 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:55,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:55,197 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:55,197 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:55,197 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:55,198 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:55,201 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:55,201 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:55,202 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:55,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:55,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,203 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 08:54:55,203 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:55,203 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-08 08:54:55,203 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-08 08:54:55,204 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,204 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,210 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:55,210 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:55,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:55,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:55,211 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:55,211 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:55,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,211 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:55,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,211 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,211 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-08 08:54:55,211 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:55,211 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-08 08:54:55,212 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:55,212 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-08 08:54:55,212 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:55,212 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:55,212 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:55,212 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. started... 2023-02-08 08:54:55,212 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 201d231772cef3ab7c7efad6d0fb3900 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:55,623 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101e56738c319264a49b659ce43c940695f to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101e56738c319264a49b659ce43c940695f 2023-02-08 08:54:56,030 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/25e1df9f207342b791b6f20fa9bf42d4 2023-02-08 08:54:56,035 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/25e1df9f207342b791b6f20fa9bf42d4 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/25e1df9f207342b791b6f20fa9bf42d4 2023-02-08 08:54:56,039 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/25e1df9f207342b791b6f20fa9bf42d4, entries=1, sequenceid=29, filesize=5.0 K 2023-02-08 08:54:56,040 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 201d231772cef3ab7c7efad6d0fb3900 in 828ms, sequenceid=29, compaction requested=true 2023-02-08 08:54:56,040 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 201d231772cef3ab7c7efad6d0fb3900: 2023-02-08 08:54:56,040 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:56,040 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:56,040 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:56,040 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,040 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-08 08:54:56,040 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-08 08:54:56,041 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,041 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:56,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:56,042 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,042 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:56,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:56,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:56,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:56,043 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-08 08:54:56,043 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@660fc26c[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:56,043 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-08 08:54:56,043 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,044 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,044 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-08 08:54:56,044 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-08 08:54:56,044 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:56,044 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,045 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:56,046 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,046 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-08 08:54:56,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:56,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:56,046 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:56,046 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-08 08:54:56,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:56,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:56,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:56,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:56,048 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-08 08:54:56,048 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-08 08:54:56,048 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:56,048 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:56,048 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,048 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:56,051 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,051 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:56,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,051 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:56,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:56,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:56,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:56,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:56,051 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:56,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,052 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:56,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,054 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,054 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,054 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,054 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,054 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:56,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 08:54:56,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:56,055 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:56,055 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-08 08:54:56,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:56,055 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:56,054 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:56,055 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,055 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:56,055 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,055 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:56,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:56,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:56,155 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:56,155 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:56,156 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:56,156 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:56,159 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:56,159 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:56,160 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:56,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:56,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,162 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 08:54:56,162 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:56,162 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-08 08:54:56,162 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-08 08:54:56,162 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,162 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,163 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,163 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:56,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:56,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:56,164 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,164 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:56,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,164 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:56,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,165 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,165 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-08 08:54:56,165 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,165 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-08 08:54:56,165 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:56,165 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-08 08:54:56,165 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:56,165 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:56,165 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:56,165 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. started... 2023-02-08 08:54:56,165 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 201d231772cef3ab7c7efad6d0fb3900 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:56,580 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601031880b32e666b4f4497946740c9a670bd to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601031880b32e666b4f4497946740c9a670bd 2023-02-08 08:54:56,586 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/ec0e8684abf047d6adf9f209958888d9 2023-02-08 08:54:56,590 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/ec0e8684abf047d6adf9f209958888d9 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/ec0e8684abf047d6adf9f209958888d9 2023-02-08 08:54:56,593 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/ec0e8684abf047d6adf9f209958888d9, entries=1, sequenceid=33, filesize=5.0 K 2023-02-08 08:54:56,594 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 201d231772cef3ab7c7efad6d0fb3900 in 429ms, sequenceid=33, compaction requested=true 2023-02-08 08:54:56,594 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 201d231772cef3ab7c7efad6d0fb3900: 2023-02-08 08:54:56,594 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:56,594 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:56,594 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:56,594 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,594 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-08 08:54:56,594 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-08 08:54:56,596 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,596 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:56,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:56,596 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,596 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:56,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:56,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:56,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:56,598 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-08 08:54:56,598 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47ce1af8[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:56,598 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-08 08:54:56,598 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,600 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,601 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-08 08:54:56,601 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-08 08:54:56,601 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:56,601 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,601 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:56,602 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,602 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-08 08:54:56,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:56,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:56,602 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:56,602 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-08 08:54:56,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:56,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:56,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:56,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:56,604 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-08 08:54:56,604 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-08 08:54:56,604 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:56,604 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:56,604 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,604 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:56,606 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,606 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:56,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:56,606 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:56,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:56,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:56,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:56,606 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:56,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,607 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:56,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,609 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,609 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:56,610 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,610 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:56,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:56,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:56,610 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,610 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 08:54:56,610 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:56,610 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,610 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-08 08:54:56,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:56,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:56,610 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:56,610 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:56,610 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:56,610 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,610 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,610 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,710 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:56,711 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:56,711 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:56,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:56,715 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:56,715 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:56,716 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:56,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:56,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,717 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 08:54:56,718 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:56,718 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-08 08:54:56,718 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-08 08:54:56,718 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,718 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,720 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:56,720 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:56,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:56,721 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:56,721 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:56,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,721 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:56,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,721 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,722 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-08 08:54:56,722 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:56,722 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-08 08:54:56,722 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:56,722 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-08 08:54:56,728 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:56,728 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:56,728 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:56,728 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. started... 2023-02-08 08:54:56,728 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 201d231772cef3ab7c7efad6d0fb3900 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:56,741 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907521dfe0bae844c35b677c680d071e8c5 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20150907521dfe0bae844c35b677c680d071e8c5 2023-02-08 08:54:57,147 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/2f68b6b27ee84af3a36dfd2a557dd3d7 2023-02-08 08:54:57,152 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/2f68b6b27ee84af3a36dfd2a557dd3d7 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/2f68b6b27ee84af3a36dfd2a557dd3d7 2023-02-08 08:54:57,155 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/2f68b6b27ee84af3a36dfd2a557dd3d7, entries=1, sequenceid=37, filesize=5.0 K 2023-02-08 08:54:57,156 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 201d231772cef3ab7c7efad6d0fb3900 in 428ms, sequenceid=37, compaction requested=true 2023-02-08 08:54:57,156 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 201d231772cef3ab7c7efad6d0fb3900: 2023-02-08 08:54:57,156 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:57,156 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:57,156 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:57,156 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:57,156 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-08 08:54:57,156 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-08 08:54:57,158 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,158 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:57,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:57,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:57,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:57,158 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,158 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:57,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:57,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:57,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:57,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:57,159 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-08 08:54:57,160 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@a91b3bd[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:57,160 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-08 08:54:57,160 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,161 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,161 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-08 08:54:57,161 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-08 08:54:57,161 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:57,161 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:57,161 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:57,163 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-08 08:54:57,163 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:57,163 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:57,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:57,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:57,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:57,163 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-08 08:54:57,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:57,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:57,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:57,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:57,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:57,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:57,165 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-08 08:54:57,165 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-08 08:54:57,165 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:57,165 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:57,165 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,165 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:57,168 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,168 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:57,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,168 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:57,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:57,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:57,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:57,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:57,168 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:57,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:57,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,169 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:57,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:57,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:57,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:57,171 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:57,171 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,171 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:57,171 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:57,171 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-08 08:54:57,171 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:57,171 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 08:54:57,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:57,171 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:57,171 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:57,172 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:57,172 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,172 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,172 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:57,172 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:57,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:57,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:57,272 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:57,272 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:57,272 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:57,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:57,276 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:57,276 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:57,277 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:54:57,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-08 08:54:57,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,278 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 08:54:57,278 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:57,278 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-08 08:54:57,278 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-08 08:54:57,279 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,279 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,280 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:57,280 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:57,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:57,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:57,280 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:57,280 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:54:57,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,281 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:54:57,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,281 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,281 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-08 08:54:57,281 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:57,281 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-08 08:54:57,281 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:54:57,281 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-08 08:54:57,282 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:54:57,282 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:54:57,282 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:57,282 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. started... 2023-02-08 08:54:57,282 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 201d231772cef3ab7c7efad6d0fb3900 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:54:57,696 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120ca5aaaea897e4ca89ffa0e8f7ac023c0 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120ca5aaaea897e4ca89ffa0e8f7ac023c0 2023-02-08 08:54:58,101 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/3a771d8e74714a32a9cff41e0a47b9b8 2023-02-08 08:54:58,105 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/3a771d8e74714a32a9cff41e0a47b9b8 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/3a771d8e74714a32a9cff41e0a47b9b8 2023-02-08 08:54:58,109 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/3a771d8e74714a32a9cff41e0a47b9b8, entries=1, sequenceid=41, filesize=5.0 K 2023-02-08 08:54:58,110 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 201d231772cef3ab7c7efad6d0fb3900 in 828ms, sequenceid=41, compaction requested=true 2023-02-08 08:54:58,110 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 201d231772cef3ab7c7efad6d0fb3900: 2023-02-08 08:54:58,110 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:58,110 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:54:58,110 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:54:58,110 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:58,110 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-08 08:54:58,110 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-08 08:54:58,112 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:58,112 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:58,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:58,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:58,112 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,112 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:54:58,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:58,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:58,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:58,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:58,113 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-08 08:54:58,113 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@391175ef[Count = 0] remaining members to acquire global barrier 2023-02-08 08:54:58,113 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-08 08:54:58,113 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,115 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,115 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-08 08:54:58,115 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-08 08:54:58,115 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:54:58,115 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:58,115 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:54:58,117 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:58,117 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-08 08:54:58,117 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:58,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:58,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:58,117 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-08 08:54:58,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:58,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:58,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:58,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:58,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:58,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:58,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:54:58,119 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-08 08:54:58,119 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-08 08:54:58,119 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:54:58,119 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:54:58,119 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,119 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:54:58,120 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,120 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:54:58,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:54:58,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,120 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:58,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:54:58,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:58,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:58,120 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:58,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:54:58,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,121 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:58,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:58,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:54:58,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:58,123 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:58,123 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:54:58,123 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:54:58,123 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:54:58,123 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-08 08:54:58,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:54:58,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:54:58,123 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,124 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:54:58,124 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,124 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-08 08:54:58,123 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:54:58,124 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:54:58,124 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-08 08:54:58,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:54:58,124 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,124 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:54:58,224 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:54:58,224 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:58,225 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:58,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:54:58,227 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37869] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:54:58,228 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-08 08:54:58,229 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-08 08:54:58,229 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:54:58,229 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675847098229 2023-02-08 08:54:58,233 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 08:54:58,233 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-08 08:54:58,233 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 08:54:58,234 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] zookeeper.ReadOnlyZKClient(139): Connect 0x0e54346f to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:54:58,238 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4fb25c4e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:54:58,239 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:54:58,678 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103487595c1d6c44438a0e5eb5902646dbf to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103487595c1d6c44438a0e5eb5902646dbf 2023-02-08 08:54:58,678 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 08:54:58,691 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:54:58,692 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:32804, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:54:58,697 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:58,697 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:58,698 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 08:54:58,699 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47760, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 08:54:58,706 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/c7cd4c55948e406ca9e46535deed3058 first=Optional[r3] last=Optional[r6] 2023-02-08 08:54:58,707 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/c7cd4c55948e406ca9e46535deed3058}] 2023-02-08 08:54:58,711 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/c7cd4c55948e406ca9e46535deed3058 for inclusion in 201d231772cef3ab7c7efad6d0fb3900/family1 2023-02-08 08:54:58,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-08 08:54:58,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 08:54:58,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for 201d231772cef3ab7c7efad6d0fb3900: 2023-02-08 08:54:58,714 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/c7cd4c55948e406ca9e46535deed3058 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__gkpqgkl9ht7lvapel73166isjcr440sobvth17s3ho2c8i9sjj9q8uavbrhthg05/family1/c7cd4c55948e406ca9e46535deed3058 2023-02-08 08:54:58,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__gkpqgkl9ht7lvapel73166isjcr440sobvth17s3ho2c8i9sjj9q8uavbrhthg05/family1/c7cd4c55948e406ca9e46535deed3058 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/e3cbe50e766b417e96a23a1656679644_SeqId_44_ 2023-02-08 08:54:58,716 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__gkpqgkl9ht7lvapel73166isjcr440sobvth17s3ho2c8i9sjj9q8uavbrhthg05/family1/c7cd4c55948e406ca9e46535deed3058 into 201d231772cef3ab7c7efad6d0fb3900/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/e3cbe50e766b417e96a23a1656679644_SeqId_44_ - updating store file list. 2023-02-08 08:54:58,719 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/e3cbe50e766b417e96a23a1656679644_SeqId_44_ into 201d231772cef3ab7c7efad6d0fb3900/family1 2023-02-08 08:54:58,720 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__gkpqgkl9ht7lvapel73166isjcr440sobvth17s3ho2c8i9sjj9q8uavbrhthg05/family1/c7cd4c55948e406ca9e46535deed3058 into 201d231772cef3ab7c7efad6d0fb3900/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/e3cbe50e766b417e96a23a1656679644_SeqId_44_) 2023-02-08 08:54:58,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__gkpqgkl9ht7lvapel73166isjcr440sobvth17s3ho2c8i9sjj9q8uavbrhthg05/family1/c7cd4c55948e406ca9e46535deed3058 2023-02-08 08:54:58,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-08 08:54:58,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-08 08:54:58,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(1912): 201d231772cef3ab7c7efad6d0fb3900/family1 is initiating minor compaction 2023-02-08 08:54:58,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@54e5982b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:58,723 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 201d231772cef3ab7c7efad6d0fb3900/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:58,723 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/72bbd8868af749a2a331b94d040012cc, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/78aef3deed474e50a30b1fd7ce9a2ae9, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/32cc67c20da34e3999f16ed8b12d0507, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/a59d939ea0c24c11b09c8550aaecc175, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/5eb1bd4950a644788e26c4aa4d53b3cf, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/25e1df9f207342b791b6f20fa9bf42d4, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/ec0e8684abf047d6adf9f209958888d9, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/2f68b6b27ee84af3a36dfd2a557dd3d7, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/3a771d8e74714a32a9cff41e0a47b9b8, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/e3cbe50e766b417e96a23a1656679644_SeqId_44_] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp, totalSize=50.3 K 2023-02-08 08:54:58,723 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 72bbd8868af749a2a331b94d040012cc, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-08 08:54:58,723 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 78aef3deed474e50a30b1fd7ce9a2ae9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-08 08:54:58,723 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 32cc67c20da34e3999f16ed8b12d0507, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-08 08:54:58,723 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting a59d939ea0c24c11b09c8550aaecc175, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-08 08:54:58,723 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 5eb1bd4950a644788e26c4aa4d53b3cf, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-08 08:54:58,723 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 25e1df9f207342b791b6f20fa9bf42d4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-08 08:54:58,723 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting ec0e8684abf047d6adf9f209958888d9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-08 08:54:58,724 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 2f68b6b27ee84af3a36dfd2a557dd3d7, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-08 08:54:58,724 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 3a771d8e74714a32a9cff41e0a47b9b8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-08 08:54:58,724 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting e3cbe50e766b417e96a23a1656679644_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-08 08:54:58,725 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 08:54:58,725 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:58,727 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512283a08d51fed9541569b3436885aaed6a8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512283a08d51fed9541569b3436885aaed6a8 2023-02-08 08:54:58,728 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231c45f8bf6fd404daebd7bd8042e8f102c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151231c45f8bf6fd404daebd7bd8042e8f102c 2023-02-08 08:54:58,729 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101e56738c319264a49b659ce43c940695f to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160101e56738c319264a49b659ce43c940695f 2023-02-08 08:54:58,729 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601031880b32e666b4f4497946740c9a670bd to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601031880b32e666b4f4497946740c9a670bd 2023-02-08 08:54:58,729 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-02-08 08:54:58,730 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-08 08:54:58,730 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:54:58,730 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:54:58,730 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:54:58,730 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:54:58,730 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:54:58,730 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:54:58,730 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:54:58,730 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:54:58,757 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] throttle.PressureAwareThroughputController(145): 201d231772cef3ab7c7efad6d0fb3900#family1#compaction#72 average throughput is 1.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-08 08:54:59,572 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/df84577676104e03beea41a6598f7d33 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/df84577676104e03beea41a6598f7d33 2023-02-08 08:54:59,576 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120560d229c8409b4a5885357a22f1c57087 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120560d229c8409b4a5885357a22f1c57087 2023-02-08 08:54:59,577 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 08:54:59,577 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 201d231772cef3ab7c7efad6d0fb3900/family1 of 201d231772cef3ab7c7efad6d0fb3900 into df84577676104e03beea41a6598f7d33(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:54:59,577 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 201d231772cef3ab7c7efad6d0fb3900: 2023-02-08 08:54:59,577 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1675846498723; duration=0sec 2023-02-08 08:54:59,577 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:59,592 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:59,593 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:59,599 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/15a9b40c65c04a1d977d54b6b1f39581 first=Optional[r00] last=Optional[r2] 2023-02-08 08:54:59,600 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/15a9b40c65c04a1d977d54b6b1f39581}] 2023-02-08 08:54:59,602 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/15a9b40c65c04a1d977d54b6b1f39581 for inclusion in 201d231772cef3ab7c7efad6d0fb3900/family1 2023-02-08 08:54:59,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-08 08:54:59,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 08:54:59,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for 201d231772cef3ab7c7efad6d0fb3900: 2023-02-08 08:54:59,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/15a9b40c65c04a1d977d54b6b1f39581 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__2s9d5f5t670ge1gfee9hshnfo746vkgdrlh32v2m1jf6uqcfpa0p6hhlje7gh51b/family1/15a9b40c65c04a1d977d54b6b1f39581 2023-02-08 08:54:59,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__2s9d5f5t670ge1gfee9hshnfo746vkgdrlh32v2m1jf6uqcfpa0p6hhlje7gh51b/family1/15a9b40c65c04a1d977d54b6b1f39581 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/ba500821a10d4b23979e82d5de5bfeb9_SeqId_47_ 2023-02-08 08:54:59,609 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__2s9d5f5t670ge1gfee9hshnfo746vkgdrlh32v2m1jf6uqcfpa0p6hhlje7gh51b/family1/15a9b40c65c04a1d977d54b6b1f39581 into 201d231772cef3ab7c7efad6d0fb3900/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/ba500821a10d4b23979e82d5de5bfeb9_SeqId_47_ - updating store file list. 2023-02-08 08:54:59,612 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/ba500821a10d4b23979e82d5de5bfeb9_SeqId_47_ into 201d231772cef3ab7c7efad6d0fb3900/family1 2023-02-08 08:54:59,612 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__2s9d5f5t670ge1gfee9hshnfo746vkgdrlh32v2m1jf6uqcfpa0p6hhlje7gh51b/family1/15a9b40c65c04a1d977d54b6b1f39581 into 201d231772cef3ab7c7efad6d0fb3900/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/ba500821a10d4b23979e82d5de5bfeb9_SeqId_47_) 2023-02-08 08:54:59,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__2s9d5f5t670ge1gfee9hshnfo746vkgdrlh32v2m1jf6uqcfpa0p6hhlje7gh51b/family1/15a9b40c65c04a1d977d54b6b1f39581 2023-02-08 08:54:59,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-08 08:54:59,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-08 08:54:59,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(1912): 201d231772cef3ab7c7efad6d0fb3900/family1 is initiating minor compaction (all files) 2023-02-08 08:54:59,614 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@57be169f; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:54:59,614 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 201d231772cef3ab7c7efad6d0fb3900/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:54:59,614 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/257e1b0da8504edda13d06572114663a, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/df84577676104e03beea41a6598f7d33, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/ba500821a10d4b23979e82d5de5bfeb9_SeqId_47_] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp, totalSize=388.1 K 2023-02-08 08:54:59,614 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 257e1b0da8504edda13d06572114663a, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878836535 2023-02-08 08:54:59,614 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting df84577676104e03beea41a6598f7d33, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621236535 2023-02-08 08:54:59,615 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting ba500821a10d4b23979e82d5de5bfeb9_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878836535 2023-02-08 08:54:59,615 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 08:54:59,616 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:59,617 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130149cab94b7d74f69a099caec265c49a4 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130149cab94b7d74f69a099caec265c49a4 2023-02-08 08:54:59,618 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512015ff1e733d1ac437dafbbeff0e2c0723c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512015ff1e733d1ac437dafbbeff0e2c0723c 2023-02-08 08:54:59,620 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205a68e59eaa2f049faaf1b95010df1912c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205a68e59eaa2f049faaf1b95010df1912c 2023-02-08 08:54:59,620 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-02-08 08:54:59,620 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-08 08:54:59,620 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:54:59,620 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 08:54:59,620 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:54:59,621 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-08 08:54:59,623 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.3950 sec 2023-02-08 08:54:59,637 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/72bbd8868af749a2a331b94d040012cc, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/78aef3deed474e50a30b1fd7ce9a2ae9, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/32cc67c20da34e3999f16ed8b12d0507, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/a59d939ea0c24c11b09c8550aaecc175, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/5eb1bd4950a644788e26c4aa4d53b3cf, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/25e1df9f207342b791b6f20fa9bf42d4, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/ec0e8684abf047d6adf9f209958888d9, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/2f68b6b27ee84af3a36dfd2a557dd3d7, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/3a771d8e74714a32a9cff41e0a47b9b8, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/e3cbe50e766b417e96a23a1656679644_SeqId_44_] to archive 2023-02-08 08:54:59,638 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] throttle.PressureAwareThroughputController(145): 201d231772cef3ab7c7efad6d0fb3900#family1#compaction#73 average throughput is 11.98 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-08 08:54:59,638 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:54:59,640 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/72bbd8868af749a2a331b94d040012cc to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/72bbd8868af749a2a331b94d040012cc 2023-02-08 08:54:59,641 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/78aef3deed474e50a30b1fd7ce9a2ae9 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/78aef3deed474e50a30b1fd7ce9a2ae9 2023-02-08 08:54:59,643 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/32cc67c20da34e3999f16ed8b12d0507 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/32cc67c20da34e3999f16ed8b12d0507 2023-02-08 08:54:59,644 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/a59d939ea0c24c11b09c8550aaecc175 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/a59d939ea0c24c11b09c8550aaecc175 2023-02-08 08:54:59,645 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/5eb1bd4950a644788e26c4aa4d53b3cf to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/5eb1bd4950a644788e26c4aa4d53b3cf 2023-02-08 08:54:59,647 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/25e1df9f207342b791b6f20fa9bf42d4 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/25e1df9f207342b791b6f20fa9bf42d4 2023-02-08 08:54:59,648 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/ec0e8684abf047d6adf9f209958888d9 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/ec0e8684abf047d6adf9f209958888d9 2023-02-08 08:54:59,649 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/2f68b6b27ee84af3a36dfd2a557dd3d7 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/2f68b6b27ee84af3a36dfd2a557dd3d7 2023-02-08 08:54:59,650 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/3a771d8e74714a32a9cff41e0a47b9b8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/3a771d8e74714a32a9cff41e0a47b9b8 2023-02-08 08:54:59,653 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/e3cbe50e766b417e96a23a1656679644_SeqId_44_ to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/e3cbe50e766b417e96a23a1656679644_SeqId_44_ 2023-02-08 08:54:59,664 DEBUG [hfile_cleaner-dir-scan-pool-0] cleaner.CleanerChore(483): Couldn't delete 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1' yet because it isn't empty w/exception. org.apache.hadoop.fs.PathIsNotEmptyDirectoryException: ``/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1 is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) ': Directory is not empty: `/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1 is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121) at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1595) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:881) at org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:878) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:878) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$4(CleanerChore.java:453) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.deleteAction(CleanerChore.java:478) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$traverseAndDelete$5(CleanerChore.java:453) at org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68) at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) at org.apache.hadoop.hbase.util.FutureUtils.addListener(FutureUtils.java:61) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:439) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$null$2(CleanerChore.java:432) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.PathIsNotEmptyDirectoryException): `/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1 is non empty': Directory is not empty at org.apache.hadoop.hdfs.server.namenode.FSDirDeleteOp.delete(FSDirDeleteOp.java:115) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:2873) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.delete(NameNodeRpcServer.java:1101) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.delete(ClientNamenodeProtocolServerSideTranslatorPB.java:648) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:507) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1034) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:994) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:922) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1893) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2833) at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1540) at org.apache.hadoop.ipc.Client.call(Client.java:1486) at org.apache.hadoop.ipc.Client.call(Client.java:1385) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) at com.sun.proxy.$Proxy29.delete(Unknown Source) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.delete(ClientNamenodeProtocolTranslatorPB.java:572) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157) at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359) at com.sun.proxy.$Proxy30.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:361) at com.sun.proxy.$Proxy33.delete(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:1593) ... 17 more 2023-02-08 08:54:59,708 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:54:59,708 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:54:59,709 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-08 08:54:59,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:55:00,120 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-08 08:55:00,121 DEBUG [PEWorker-5] util.FSTableDescriptors(519): Deleted hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-02-08 08:55:00,121 INFO [PEWorker-5] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-08 08:55:00,123 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-02-08 08:55:00,125 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=201d231772cef3ab7c7efad6d0fb3900, REOPEN/MOVE}] 2023-02-08 08:55:00,126 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=201d231772cef3ab7c7efad6d0fb3900, REOPEN/MOVE 2023-02-08 08:55:00,126 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=201d231772cef3ab7c7efad6d0fb3900, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:00,127 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675846500126"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846500126"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846500126"}]},"ts":"1675846500126"} 2023-02-08 08:55:00,128 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure 201d231772cef3ab7c7efad6d0fb3900, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:55:00,280 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:55:00,280 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 201d231772cef3ab7c7efad6d0fb3900, disabling compactions & flushes 2023-02-08 08:55:00,280 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:55:00,856 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/.tmp/family1/ba3719d9c18b41059957057a6d7f68b6 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/ba3719d9c18b41059957057a6d7f68b6 2023-02-08 08:55:00,860 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 201d231772cef3ab7c7efad6d0fb3900/family1 of 201d231772cef3ab7c7efad6d0fb3900 into ba3719d9c18b41059957057a6d7f68b6(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 08:55:00,860 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 201d231772cef3ab7c7efad6d0fb3900: 2023-02-08 08:55:00,860 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:55:00,860 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1675846499613; duration=1sec 2023-02-08 08:55:00,860 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:55:00,860 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. after waiting 0 ms 2023-02-08 08:55:00,860 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:55:00,860 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:55:00,863 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/257e1b0da8504edda13d06572114663a, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/df84577676104e03beea41a6598f7d33, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/ba500821a10d4b23979e82d5de5bfeb9_SeqId_47_] to archive 2023-02-08 08:55:00,864 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:00,866 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/257e1b0da8504edda13d06572114663a to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/257e1b0da8504edda13d06572114663a 2023-02-08 08:55:00,867 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/df84577676104e03beea41a6598f7d33 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/df84577676104e03beea41a6598f7d33 2023-02-08 08:55:00,868 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/ba500821a10d4b23979e82d5de5bfeb9_SeqId_47_ to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/ba500821a10d4b23979e82d5de5bfeb9_SeqId_47_ 2023-02-08 08:55:00,872 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-08 08:55:00,873 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:55:00,873 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 201d231772cef3ab7c7efad6d0fb3900: Waiting for close lock at 1675846500280Running coprocessor pre-close hooks at 1675846500280Disabling compacts and flushes for region at 1675846500280Disabling writes for close at 1675846500860 (+580 ms)Writing region close event to WAL at 1675846500870 (+10 ms)Running coprocessor post-close hooks at 1675846500872 (+2 ms)Closed at 1675846500873 (+1 ms) 2023-02-08 08:55:00,873 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 201d231772cef3ab7c7efad6d0fb3900 to self. 2023-02-08 08:55:00,874 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:55:00,874 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=201d231772cef3ab7c7efad6d0fb3900, regionState=CLOSED 2023-02-08 08:55:00,875 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675846500874"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846500874"}]},"ts":"1675846500874"} 2023-02-08 08:55:00,877 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-02-08 08:55:00,877 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure 201d231772cef3ab7c7efad6d0fb3900, server=jenkins-hbase4.apache.org,42239,1675846413348 in 747 msec 2023-02-08 08:55:00,877 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=201d231772cef3ab7c7efad6d0fb3900, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=true 2023-02-08 08:55:01,027 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=201d231772cef3ab7c7efad6d0fb3900, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:01,028 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675846501027"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846501027"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846501027"}]},"ts":"1675846501027"} 2023-02-08 08:55:01,029 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure 201d231772cef3ab7c7efad6d0fb3900, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:55:01,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:55:01,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 201d231772cef3ab7c7efad6d0fb3900, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:55:01,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:55:01,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:01,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:55:01,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:55:01,185 INFO [StoreOpener-201d231772cef3ab7c7efad6d0fb3900-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:55:01,185 DEBUG [StoreOpener-201d231772cef3ab7c7efad6d0fb3900-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1 2023-02-08 08:55:01,186 DEBUG [StoreOpener-201d231772cef3ab7c7efad6d0fb3900-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1 2023-02-08 08:55:01,186 INFO [StoreOpener-201d231772cef3ab7c7efad6d0fb3900-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 201d231772cef3ab7c7efad6d0fb3900 columnFamilyName family1 2023-02-08 08:55:01,190 DEBUG [StoreOpener-201d231772cef3ab7c7efad6d0fb3900-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/ba3719d9c18b41059957057a6d7f68b6 2023-02-08 08:55:01,190 INFO [StoreOpener-201d231772cef3ab7c7efad6d0fb3900-1] regionserver.HStore(310): Store=201d231772cef3ab7c7efad6d0fb3900/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:01,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:55:01,192 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:55:01,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:55:01,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 201d231772cef3ab7c7efad6d0fb3900; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64847485, jitterRate=-0.03369717299938202}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 08:55:01,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 201d231772cef3ab7c7efad6d0fb3900: Running coprocessor pre-open hook at 1675846501184Writing region info on filesystem at 1675846501184Initializing all the Stores at 1675846501184Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675846501184Cleaning up temporary data from old regions at 1675846501193 (+9 ms)Cleaning up detritus from prior splits at 1675846501193Running coprocessor post-open hooks at 1675846501195 (+2 ms)Region opened successfully at 1675846501195 2023-02-08 08:55:01,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900., pid=63, masterSystemTime=1675846501181 2023-02-08 08:55:01,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:55:01,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:55:01,197 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=201d231772cef3ab7c7efad6d0fb3900, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:01,197 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675846501197"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846501197"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846501197"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846501197"}]},"ts":"1675846501197"} 2023-02-08 08:55:01,199 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-02-08 08:55:01,199 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure 201d231772cef3ab7c7efad6d0fb3900, server=jenkins-hbase4.apache.org,42239,1675846413348 in 169 msec 2023-02-08 08:55:01,201 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-02-08 08:55:01,201 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=201d231772cef3ab7c7efad6d0fb3900, REOPEN/MOVE in 1.0740 sec 2023-02-08 08:55:01,202 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-02-08 08:55:01,202 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.0780 sec 2023-02-08 08:55:01,203 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.4940 sec 2023-02-08 08:55:01,204 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:01,204 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:01,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-02-08 08:55:01,205 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:01,205 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:01,205 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:01,205 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:01,206 INFO [Listener at localhost/40763] compactions.TestMobCompactor(278): alter status finished 2023-02-08 08:55:01,207 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37869] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:55:01,208 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-08 08:55:01,208 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-08 08:55:01,209 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:55:01,209 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675847101209 2023-02-08 08:55:01,209 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 08:55:01,210 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-08 08:55:01,210 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 08:55:01,210 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] zookeeper.ReadOnlyZKClient(139): Connect 0x25aa404e to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:55:01,214 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@14f224e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:55:01,215 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:55:01,215 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:55:01,215 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:55:01,215 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:55:01,215 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:55:02,037 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128863b4b0bc68a470e8d2998988decd42c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151128863b4b0bc68a470e8d2998988decd42c 2023-02-08 08:55:02,038 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 08:55:02,050 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:55:02,051 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:32806, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:55:02,056 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:02,056 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:02,057 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 08:55:02,058 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47776, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 08:55:02,065 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/0f0918b348e347c4b122a5260cbc6521 first=Optional[r05] last=Optional[r8] 2023-02-08 08:55:02,066 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=51 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/0f0918b348e347c4b122a5260cbc6521}] 2023-02-08 08:55:02,069 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/0f0918b348e347c4b122a5260cbc6521 for inclusion in 201d231772cef3ab7c7efad6d0fb3900/family1 2023-02-08 08:55:02,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-08 08:55:02,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 08:55:02,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for 201d231772cef3ab7c7efad6d0fb3900: 2023-02-08 08:55:02,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/0f0918b348e347c4b122a5260cbc6521 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__d4v4ld9p0gt076qov1e26jn993sqhl914a6npdpedfgrlukme67hl1a85ltehlak/family1/0f0918b348e347c4b122a5260cbc6521 2023-02-08 08:55:02,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__d4v4ld9p0gt076qov1e26jn993sqhl914a6npdpedfgrlukme67hl1a85ltehlak/family1/0f0918b348e347c4b122a5260cbc6521 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/6a744ed6f4834c44b9403b28d2ce37b3_SeqId_53_ 2023-02-08 08:55:02,075 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__d4v4ld9p0gt076qov1e26jn993sqhl914a6npdpedfgrlukme67hl1a85ltehlak/family1/0f0918b348e347c4b122a5260cbc6521 into 201d231772cef3ab7c7efad6d0fb3900/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/6a744ed6f4834c44b9403b28d2ce37b3_SeqId_53_ - updating store file list. 2023-02-08 08:55:02,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/6a744ed6f4834c44b9403b28d2ce37b3_SeqId_53_ into 201d231772cef3ab7c7efad6d0fb3900/family1 2023-02-08 08:55:02,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__d4v4ld9p0gt076qov1e26jn993sqhl914a6npdpedfgrlukme67hl1a85ltehlak/family1/0f0918b348e347c4b122a5260cbc6521 into 201d231772cef3ab7c7efad6d0fb3900/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/6a744ed6f4834c44b9403b28d2ce37b3_SeqId_53_) 2023-02-08 08:55:02,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__d4v4ld9p0gt076qov1e26jn993sqhl914a6npdpedfgrlukme67hl1a85ltehlak/family1/0f0918b348e347c4b122a5260cbc6521 2023-02-08 08:55:02,080 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 08:55:02,081 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:02,082 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120ca5aaaea897e4ca89ffa0e8f7ac023c0 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151120ca5aaaea897e4ca89ffa0e8f7ac023c0 2023-02-08 08:55:02,083 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511284cee06780f5241e6af2751e6070e2b9c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511284cee06780f5241e6af2751e6070e2b9c 2023-02-08 08:55:02,083 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-02-08 08:55:02,083 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-08 08:55:02,083 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:55:02,083 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:55:02,083 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:55:02,084 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 08:55:02,084 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:55:02,084 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-08 08:55:02,085 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 877 msec 2023-02-08 08:55:02,130 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:02,130 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:02,131 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-02-08 08:55:02,132 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:55:02,540 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-02-08 08:55:02,541 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-08 08:55:02,541 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-02-08 08:55:02,542 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-02-08 08:55:02,544 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=201d231772cef3ab7c7efad6d0fb3900, REOPEN/MOVE}] 2023-02-08 08:55:02,545 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=201d231772cef3ab7c7efad6d0fb3900, REOPEN/MOVE 2023-02-08 08:55:02,545 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=201d231772cef3ab7c7efad6d0fb3900, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:02,546 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675846502545"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846502545"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846502545"}]},"ts":"1675846502545"} 2023-02-08 08:55:02,547 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure 201d231772cef3ab7c7efad6d0fb3900, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:55:02,698 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:55:02,698 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 201d231772cef3ab7c7efad6d0fb3900, disabling compactions & flushes 2023-02-08 08:55:02,698 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:55:02,698 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:55:02,698 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. after waiting 0 ms 2023-02-08 08:55:02,698 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:55:02,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-08 08:55:02,704 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:55:02,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 201d231772cef3ab7c7efad6d0fb3900: Waiting for close lock at 1675846502698Running coprocessor pre-close hooks at 1675846502698Disabling compacts and flushes for region at 1675846502698Disabling writes for close at 1675846502698Writing region close event to WAL at 1675846502701 (+3 ms)Running coprocessor post-close hooks at 1675846502704 (+3 ms)Closed at 1675846502704 2023-02-08 08:55:02,704 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 201d231772cef3ab7c7efad6d0fb3900 to self. 2023-02-08 08:55:02,705 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:55:02,705 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=201d231772cef3ab7c7efad6d0fb3900, regionState=CLOSED 2023-02-08 08:55:02,705 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675846502705"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846502705"}]},"ts":"1675846502705"} 2023-02-08 08:55:02,707 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-02-08 08:55:02,708 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure 201d231772cef3ab7c7efad6d0fb3900, server=jenkins-hbase4.apache.org,42239,1675846413348 in 159 msec 2023-02-08 08:55:02,708 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=201d231772cef3ab7c7efad6d0fb3900, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=true 2023-02-08 08:55:02,858 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=201d231772cef3ab7c7efad6d0fb3900, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:02,859 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675846502858"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846502858"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846502858"}]},"ts":"1675846502858"} 2023-02-08 08:55:02,860 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure 201d231772cef3ab7c7efad6d0fb3900, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:55:03,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:55:03,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 201d231772cef3ab7c7efad6d0fb3900, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:55:03,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:55:03,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:03,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:55:03,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:55:03,015 INFO [StoreOpener-201d231772cef3ab7c7efad6d0fb3900-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:55:03,016 DEBUG [StoreOpener-201d231772cef3ab7c7efad6d0fb3900-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1 2023-02-08 08:55:03,016 DEBUG [StoreOpener-201d231772cef3ab7c7efad6d0fb3900-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1 2023-02-08 08:55:03,017 INFO [StoreOpener-201d231772cef3ab7c7efad6d0fb3900-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 201d231772cef3ab7c7efad6d0fb3900 columnFamilyName family1 2023-02-08 08:55:03,021 DEBUG [StoreOpener-201d231772cef3ab7c7efad6d0fb3900-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/6a744ed6f4834c44b9403b28d2ce37b3_SeqId_53_ 2023-02-08 08:55:03,024 DEBUG [StoreOpener-201d231772cef3ab7c7efad6d0fb3900-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/family1/ba3719d9c18b41059957057a6d7f68b6 2023-02-08 08:55:03,024 INFO [StoreOpener-201d231772cef3ab7c7efad6d0fb3900-1] regionserver.HStore(310): Store=201d231772cef3ab7c7efad6d0fb3900/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:03,025 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:55:03,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:55:03,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:55:03,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 201d231772cef3ab7c7efad6d0fb3900; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59085393, jitterRate=-0.11955903470516205}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 08:55:03,029 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 201d231772cef3ab7c7efad6d0fb3900: Running coprocessor pre-open hook at 1675846503014Writing region info on filesystem at 1675846503014Initializing all the Stores at 1675846503015 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} at 1675846503015Cleaning up temporary data from old regions at 1675846503026 (+11 ms)Cleaning up detritus from prior splits at 1675846503026Running coprocessor post-open hooks at 1675846503029 (+3 ms)Region opened successfully at 1675846503029 2023-02-08 08:55:03,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900., pid=69, masterSystemTime=1675846503011 2023-02-08 08:55:03,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:55:03,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:55:03,031 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=201d231772cef3ab7c7efad6d0fb3900, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:03,031 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1675846503031"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846503031"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846503031"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846503031"}]},"ts":"1675846503031"} 2023-02-08 08:55:03,033 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-02-08 08:55:03,033 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure 201d231772cef3ab7c7efad6d0fb3900, server=jenkins-hbase4.apache.org,42239,1675846413348 in 172 msec 2023-02-08 08:55:03,034 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-02-08 08:55:03,034 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=201d231772cef3ab7c7efad6d0fb3900, REOPEN/MOVE in 489 msec 2023-02-08 08:55:03,035 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-08 08:55:03,035 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 492 msec 2023-02-08 08:55:03,037 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 905 msec 2023-02-08 08:55:03,037 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:03,037 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:03,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-02-08 08:55:03,038 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:03,038 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:03,038 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:03,038 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:03,039 INFO [Listener at localhost/40763] compactions.TestMobCompactor(278): alter status finished 2023-02-08 08:55:03,040 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37869] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-08 08:55:03,041 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-08 08:55:03,041 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-08 08:55:03,042 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:55:03,042 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675847103042 2023-02-08 08:55:03,042 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 08:55:03,043 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-08 08:55:03,043 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 08:55:03,043 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] zookeeper.ReadOnlyZKClient(139): Connect 0x45565d27 to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:55:03,048 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@59a2e503, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:55:03,048 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:55:03,048 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:55:03,048 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:55:03,048 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:55:03,048 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:55:03,048 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:55:03,049 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:55:03,049 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-08 08:55:03,049 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-08 08:55:03,049 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:55:03,049 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-08 08:55:03,050 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 9 msec 2023-02-08 08:55:03,093 INFO [Listener at localhost/40763] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=251 (was 242) - Thread LEAK? -, OpenFileDescriptor=645 (was 606) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=210 (was 270), ProcessCount=168 (was 168), AvailableMemoryMB=4169 (was 4184) 2023-02-08 08:55:03,105 INFO [Listener at localhost/40763] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=251, OpenFileDescriptor=645, MaxFileDescriptor=60000, SystemLoadAverage=210, ProcessCount=168, AvailableMemoryMB=4169 2023-02-08 08:55:03,106 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:03,106 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:03,107 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-08 08:55:03,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-02-08 08:55:03,109 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:55:03,109 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-02-08 08:55:03,109 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:03,109 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:03,109 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:55:03,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-08 08:55:03,110 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a 2023-02-08 08:55:03,111 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a empty. 2023-02-08 08:55:03,111 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a 2023-02-08 08:55:03,111 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-02-08 08:55:03,210 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:03,210 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:03,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-08 08:55:03,411 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:03,411 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:03,412 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-08 08:55:03,518 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-02-08 08:55:03,520 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7671): creating {ENCODED => 23f89a76708cf9c5d819f0df80d3c88a, NAME => 'testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp 2023-02-08 08:55:03,712 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:03,712 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:03,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-08 08:55:03,925 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(865): Instantiated testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:03,925 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1603): Closing 23f89a76708cf9c5d819f0df80d3c88a, disabling compactions & flushes 2023-02-08 08:55:03,925 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1625): Closing region testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a. 2023-02-08 08:55:03,925 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a. 2023-02-08 08:55:03,925 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1713): Acquired close lock on testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a. after waiting 0 ms 2023-02-08 08:55:03,925 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1723): Updates disabled for region testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a. 2023-02-08 08:55:03,925 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1837): Closed testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a. 2023-02-08 08:55:03,925 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1557): Region close journal for 23f89a76708cf9c5d819f0df80d3c88a: Waiting for close lock at 1675846503925Disabling compacts and flushes for region at 1675846503925Disabling writes for close at 1675846503925Writing region close event to WAL at 1675846503925Closed at 1675846503925 2023-02-08 08:55:03,927 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:55:03,928 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846503928"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846503928"}]},"ts":"1675846503928"} 2023-02-08 08:55:03,929 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 08:55:03,930 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:55:03,930 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846503930"}]},"ts":"1675846503930"} 2023-02-08 08:55:03,931 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-02-08 08:55:03,933 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=23f89a76708cf9c5d819f0df80d3c88a, ASSIGN}] 2023-02-08 08:55:03,934 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=23f89a76708cf9c5d819f0df80d3c88a, ASSIGN 2023-02-08 08:55:03,935 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=23f89a76708cf9c5d819f0df80d3c88a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=false 2023-02-08 08:55:04,086 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=23f89a76708cf9c5d819f0df80d3c88a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:04,086 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846504086"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846504086"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846504086"}]},"ts":"1675846504086"} 2023-02-08 08:55:04,087 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure 23f89a76708cf9c5d819f0df80d3c88a, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:55:04,213 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:04,213 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:04,214 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-08 08:55:04,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a. 2023-02-08 08:55:04,241 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 23f89a76708cf9c5d819f0df80d3c88a, NAME => 'testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:55:04,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-02-08 08:55:04,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-02-08 08:55:04,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction 23f89a76708cf9c5d819f0df80d3c88a 2023-02-08 08:55:04,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:04,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 23f89a76708cf9c5d819f0df80d3c88a 2023-02-08 08:55:04,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 23f89a76708cf9c5d819f0df80d3c88a 2023-02-08 08:55:04,245 INFO [StoreOpener-23f89a76708cf9c5d819f0df80d3c88a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 23f89a76708cf9c5d819f0df80d3c88a 2023-02-08 08:55:04,246 DEBUG [StoreOpener-23f89a76708cf9c5d819f0df80d3c88a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1 2023-02-08 08:55:04,246 DEBUG [StoreOpener-23f89a76708cf9c5d819f0df80d3c88a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1 2023-02-08 08:55:04,246 INFO [StoreOpener-23f89a76708cf9c5d819f0df80d3c88a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 23f89a76708cf9c5d819f0df80d3c88a columnFamilyName f1 2023-02-08 08:55:04,247 INFO [StoreOpener-23f89a76708cf9c5d819f0df80d3c88a-1] regionserver.HStore(310): Store=23f89a76708cf9c5d819f0df80d3c88a/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:04,248 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a 2023-02-08 08:55:04,248 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a 2023-02-08 08:55:04,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 23f89a76708cf9c5d819f0df80d3c88a 2023-02-08 08:55:04,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:55:04,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 23f89a76708cf9c5d819f0df80d3c88a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66568662, jitterRate=-0.008049637079238892}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 08:55:04,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 23f89a76708cf9c5d819f0df80d3c88a: Running coprocessor pre-open hook at 1675846504244Writing region info on filesystem at 1675846504244Initializing all the Stores at 1675846504244Instantiating store for column family {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846504244Cleaning up temporary data from old regions at 1675846504249 (+5 ms)Cleaning up detritus from prior splits at 1675846504249Running coprocessor post-open hooks at 1675846504253 (+4 ms)Region opened successfully at 1675846504253 2023-02-08 08:55:04,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a., pid=73, masterSystemTime=1675846504239 2023-02-08 08:55:04,255 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a. 2023-02-08 08:55:04,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a. 2023-02-08 08:55:04,255 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=23f89a76708cf9c5d819f0df80d3c88a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:04,255 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846504255"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846504255"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846504255"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846504255"}]},"ts":"1675846504255"} 2023-02-08 08:55:04,257 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-08 08:55:04,257 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure 23f89a76708cf9c5d819f0df80d3c88a, server=jenkins-hbase4.apache.org,42239,1675846413348 in 169 msec 2023-02-08 08:55:04,259 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-02-08 08:55:04,259 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=23f89a76708cf9c5d819f0df80d3c88a, ASSIGN in 324 msec 2023-02-08 08:55:04,259 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:55:04,259 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846504259"}]},"ts":"1675846504259"} 2023-02-08 08:55:04,260 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-02-08 08:55:04,263 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:55:04,263 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1560 sec 2023-02-08 08:55:04,499 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-02-08 08:55:05,214 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:05,214 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:05,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-08 08:55:05,215 INFO [Listener at localhost/40763] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-02-08 08:55:05,215 DEBUG [Listener at localhost/40763] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-02-08 08:55:05,217 INFO [Listener at localhost/40763] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-08 08:55:05,219 INFO [Listener at localhost/40763] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-02-08 08:55:05,220 INFO [Listener at localhost/40763] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-08 08:55:05,220 INFO [Listener at localhost/40763] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-02-08 08:55:05,224 INFO [Listener at localhost/40763] regionserver.HRegion(2744): Flushing 23f89a76708cf9c5d819f0df80d3c88a 1/1 column families, dataSize=37 B heapSize=368 B 2023-02-08 08:55:05,637 INFO [Listener at localhost/40763] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302084d983c2d9a1d4e4d86dad20441c30fba to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302084d983c2d9a1d4e4d86dad20441c30fba 2023-02-08 08:55:06,042 INFO [Listener at localhost/40763] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/.tmp/f1/29ed9c3f4fe34fc9821e849e2b850777 2023-02-08 08:55:06,046 DEBUG [Listener at localhost/40763] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/.tmp/f1/29ed9c3f4fe34fc9821e849e2b850777 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777 2023-02-08 08:55:06,050 INFO [Listener at localhost/40763] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777, entries=1, sequenceid=5, filesize=4.9 K 2023-02-08 08:55:06,051 INFO [Listener at localhost/40763] regionserver.HRegion(2947): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 23f89a76708cf9c5d819f0df80d3c88a in 827ms, sequenceid=5, compaction requested=false 2023-02-08 08:55:06,052 DEBUG [Listener at localhost/40763] regionserver.HRegion(2445): Flush status journal for 23f89a76708cf9c5d819f0df80d3c88a: 2023-02-08 08:55:06,054 INFO [Listener at localhost/40763] regionserver.HRegion(2744): Flushing 23f89a76708cf9c5d819f0df80d3c88a 1/1 column families, dataSize=37 B heapSize=368 B 2023-02-08 08:55:06,407 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 08:55:06,451 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 08:55:06,451 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3f39dd85 to 127.0.0.1:61594 2023-02-08 08:55:06,451 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:55:06,453 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x38487ead to 127.0.0.1:61594 2023-02-08 08:55:06,453 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:55:06,454 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 08:55:06,455 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2736c997 to 127.0.0.1:61594 2023-02-08 08:55:06,455 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:55:06,455 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x45565d27 to 127.0.0.1:61594 2023-02-08 08:55:06,455 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:55:06,457 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 08:55:06,457 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0e54346f to 127.0.0.1:61594 2023-02-08 08:55:06,457 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:55:06,458 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 08:55:06,458 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x25aa404e to 127.0.0.1:61594 2023-02-08 08:55:06,458 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:55:06,465 INFO [Listener at localhost/40763] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208eac624d830b848a4a33bd84e29aa4daf to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230208eac624d830b848a4a33bd84e29aa4daf 2023-02-08 08:55:06,470 INFO [Listener at localhost/40763] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/.tmp/f1/b66b94af0f5a4f20931253b85ab21479 2023-02-08 08:55:06,474 DEBUG [Listener at localhost/40763] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/.tmp/f1/b66b94af0f5a4f20931253b85ab21479 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479 2023-02-08 08:55:06,478 INFO [Listener at localhost/40763] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479, entries=1, sequenceid=9, filesize=4.9 K 2023-02-08 08:55:06,478 INFO [Listener at localhost/40763] regionserver.HRegion(2947): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for 23f89a76708cf9c5d819f0df80d3c88a in 424ms, sequenceid=9, compaction requested=false 2023-02-08 08:55:06,479 DEBUG [Listener at localhost/40763] regionserver.HRegion(2445): Flush status journal for 23f89a76708cf9c5d819f0df80d3c88a: 2023-02-08 08:55:06,480 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37869] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-02-08 08:55:06,481 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-08 08:55:06,482 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-08 08:55:06,482 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:55:06,482 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675847106482 2023-02-08 08:55:06,483 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-08 08:55:06,483 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-02-08 08:55:06,483 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-02-08 08:55:06,483 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] zookeeper.ReadOnlyZKClient(139): Connect 0x7ecd2260 to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:55:06,487 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@30094a47, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:55:06,487 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230208 for table='testGetAfterCompaction' and column='f1' 2023-02-08 08:55:06,531 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:06,532 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:06,582 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:06,582 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:06,633 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:06,633 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:06,684 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:06,684 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:06,735 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:06,735 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:06,786 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:06,786 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:06,836 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:06,837 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:06,887 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:06,887 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:06,913 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208e1f9719dced3498d984698b4a2b134bf to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230208e1f9719dced3498d984698b4a2b134bf 2023-02-08 08:55:06,914 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230208 table='testGetAfterCompaction' column='f1' 2023-02-08 08:55:06,927 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:55:06,928 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40170, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:55:06,934 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:06,934 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:06,935 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 08:55:06,936 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:45866, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 08:55:06,938 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:06,938 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:06,944 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230208/f1/28c52fb18cfc4a8ba2b3026f3f962dc8 first=Optional[r1] last=Optional[r2] 2023-02-08 08:55:06,944 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row with hfile group [{f1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230208/f1/28c52fb18cfc4a8ba2b3026f3f962dc8}] 2023-02-08 08:55:06,948 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230208/f1/28c52fb18cfc4a8ba2b3026f3f962dc8 for inclusion in 23f89a76708cf9c5d819f0df80d3c88a/f1 2023-02-08 08:55:06,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-02-08 08:55:06,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 08:55:06,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for 23f89a76708cf9c5d819f0df80d3c88a: 2023-02-08 08:55:06,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230208/f1/28c52fb18cfc4a8ba2b3026f3f962dc8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testGetAfterCompaction__gsuiq8b6pd676slm0dsfogoqaci4vqn2tcus897la84km3k9196pku8g083ttalf/f1/28c52fb18cfc4a8ba2b3026f3f962dc8 2023-02-08 08:55:06,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testGetAfterCompaction__gsuiq8b6pd676slm0dsfogoqaci4vqn2tcus897la84km3k9196pku8g083ttalf/f1/28c52fb18cfc4a8ba2b3026f3f962dc8 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_ 2023-02-08 08:55:06,954 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testGetAfterCompaction__gsuiq8b6pd676slm0dsfogoqaci4vqn2tcus897la84km3k9196pku8g083ttalf/f1/28c52fb18cfc4a8ba2b3026f3f962dc8 into 23f89a76708cf9c5d819f0df80d3c88a/f1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_ - updating store file list. 2023-02-08 08:55:06,958 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_ into 23f89a76708cf9c5d819f0df80d3c88a/f1 2023-02-08 08:55:06,958 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testGetAfterCompaction__gsuiq8b6pd676slm0dsfogoqaci4vqn2tcus897la84km3k9196pku8g083ttalf/f1/28c52fb18cfc4a8ba2b3026f3f962dc8 into 23f89a76708cf9c5d819f0df80d3c88a/f1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_) 2023-02-08 08:55:06,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testGetAfterCompaction__gsuiq8b6pd676slm0dsfogoqaci4vqn2tcus897la84km3k9196pku8g083ttalf/f1/28c52fb18cfc4a8ba2b3026f3f962dc8 2023-02-08 08:55:06,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(1912): 23f89a76708cf9c5d819f0df80d3c88a/f1 is initiating minor compaction 2023-02-08 08:55:06,958 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@248d5e87; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:55:06,959 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 23f89a76708cf9c5d819f0df80d3c88a/f1 in testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a. 2023-02-08 08:55:06,959 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/.tmp, totalSize=10.0 K 2023-02-08 08:55:06,959 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting b66b94af0f5a4f20931253b85ab21479, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-08 08:55:06,959 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-08 08:55:06,961 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230208 table='testGetAfterCompaction' column='f1' 2023-02-08 08:55:06,961 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:06,963 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302084d983c2d9a1d4e4d86dad20441c30fba to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302084d983c2d9a1d4e4d86dad20441c30fba 2023-02-08 08:55:06,964 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230208eac624d830b848a4a33bd84e29aa4daf to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230208eac624d830b848a4a33bd84e29aa4daf 2023-02-08 08:55:06,964 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-02-08 08:55:06,964 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230208' 2023-02-08 08:55:06,965 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230208 for table='testGetAfterCompaction' and column='f1' 2023-02-08 08:55:06,965 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-02-08 08:55:06,965 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:55:06,966 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-08 08:55:06,967 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] throttle.PressureAwareThroughputController(145): 23f89a76708cf9c5d819f0df80d3c88a#f1#compaction#76 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-08 08:55:06,967 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 485 msec 2023-02-08 08:55:06,989 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675846506909; access_time=1675846506498; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 08:55:06,989 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:06,989 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:06,992 DEBUG [Listener at localhost/40763] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-08 08:55:06,996 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40178, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-08 08:55:06,997 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42239] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a. 2023-02-08 08:55:06,997 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42239] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-02-08 08:55:06,997 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42239] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-08 08:55:06,997 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42239] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-08 08:55:06,997 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42239] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a. because compaction request was cancelled 2023-02-08 08:55:07,049 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675846506909; access_time=1675846506498; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 08:55:07,049 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,049 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,100 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675846506909; access_time=1675846506498; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 08:55:07,100 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,100 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,151 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675846506909; access_time=1675846506498; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 08:55:07,151 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,151 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,202 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675846506909; access_time=1675846506498; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 08:55:07,202 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,202 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,252 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675846506909; access_time=1675846506498; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 08:55:07,253 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,253 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,303 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675846506909; access_time=1675846506498; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 08:55:07,303 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,304 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,354 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675846506909; access_time=1675846506498; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 08:55:07,354 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,354 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,405 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675846506909; access_time=1675846506498; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 08:55:07,405 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,405 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,456 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675846506909; access_time=1675846506498; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 08:55:07,456 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,456 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,507 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675846506909; access_time=1675846506498; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 08:55:07,507 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,507 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,557 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675846506909; access_time=1675846506498; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 08:55:07,558 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,558 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,608 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675846506909; access_time=1675846506498; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 08:55:07,608 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,608 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,659 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675846506909; access_time=1675846506498; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 08:55:07,659 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,659 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,710 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675846506909; access_time=1675846506498; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 08:55:07,710 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,710 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,761 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675846506909; access_time=1675846506498; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 08:55:07,761 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,761 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,782 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/.tmp/f1/4134d6741eeb44f6aa6f8f157ae481d1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/4134d6741eeb44f6aa6f8f157ae481d1 2023-02-08 08:55:07,786 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in 23f89a76708cf9c5d819f0df80d3c88a/f1 of 23f89a76708cf9c5d819f0df80d3c88a into 4134d6741eeb44f6aa6f8f157ae481d1(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:55:07,786 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 23f89a76708cf9c5d819f0df80d3c88a: 2023-02-08 08:55:07,786 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1675846506958; duration=0sec 2023-02-08 08:55:07,787 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:55:07,812 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1675846506909; access_time=1675846506498; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-08 08:55:07,812 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,812 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/4134d6741eeb44f6aa6f8f157ae481d1; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1675846507778; access_time=1675846506964; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,812 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506470; access_time=1675846506055; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,827 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_] to archive 2023-02-08 08:55:07,827 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:07,829 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/b66b94af0f5a4f20931253b85ab21479 2023-02-08 08:55:07,830 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_ to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/259fcfcec4c64fb08657f8b0a2d98e5d_SeqId_12_ 2023-02-08 08:55:07,862 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1675846506042; access_time=1675846505226; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,862 INFO [Listener at localhost/40763] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/4134d6741eeb44f6aa6f8f157ae481d1; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1675846507778; access_time=1675846506964; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-08 08:55:07,978 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:07,978 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:07,979 INFO [Listener at localhost/40763] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-02-08 08:55:07,983 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-02-08 08:55:07,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-02-08 08:55:07,991 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846507991"}]},"ts":"1675846507991"} 2023-02-08 08:55:07,991 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:07,992 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:07,992 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-02-08 08:55:07,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-08 08:55:07,997 INFO [PEWorker-4] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-02-08 08:55:07,998 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=23f89a76708cf9c5d819f0df80d3c88a, UNASSIGN}] 2023-02-08 08:55:07,999 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=23f89a76708cf9c5d819f0df80d3c88a, UNASSIGN 2023-02-08 08:55:08,000 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=23f89a76708cf9c5d819f0df80d3c88a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:08,000 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846508000"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846508000"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846508000"}]},"ts":"1675846508000"} 2023-02-08 08:55:08,001 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure 23f89a76708cf9c5d819f0df80d3c88a, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:55:08,092 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:08,092 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:08,093 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-08 08:55:08,153 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 23f89a76708cf9c5d819f0df80d3c88a 2023-02-08 08:55:08,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 23f89a76708cf9c5d819f0df80d3c88a, disabling compactions & flushes 2023-02-08 08:55:08,153 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a. 2023-02-08 08:55:08,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a. 2023-02-08 08:55:08,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a. after waiting 0 ms 2023-02-08 08:55:08,153 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a. 2023-02-08 08:55:08,156 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-02-08 08:55:08,157 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-02-08 08:55:08,158 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a. 2023-02-08 08:55:08,158 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 23f89a76708cf9c5d819f0df80d3c88a: Waiting for close lock at 1675846508153Running coprocessor pre-close hooks at 1675846508153Disabling compacts and flushes for region at 1675846508153Disabling writes for close at 1675846508153Writing region close event to WAL at 1675846508153Running coprocessor post-close hooks at 1675846508157 (+4 ms)Closed at 1675846508158 (+1 ms) 2023-02-08 08:55:08,159 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 23f89a76708cf9c5d819f0df80d3c88a 2023-02-08 08:55:08,159 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=23f89a76708cf9c5d819f0df80d3c88a, regionState=CLOSED 2023-02-08 08:55:08,160 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1675846508159"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846508159"}]},"ts":"1675846508159"} 2023-02-08 08:55:08,161 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-02-08 08:55:08,162 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure 23f89a76708cf9c5d819f0df80d3c88a, server=jenkins-hbase4.apache.org,42239,1675846413348 in 159 msec 2023-02-08 08:55:08,163 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-02-08 08:55:08,163 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=23f89a76708cf9c5d819f0df80d3c88a, UNASSIGN in 164 msec 2023-02-08 08:55:08,163 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846508163"}]},"ts":"1675846508163"} 2023-02-08 08:55:08,164 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-02-08 08:55:08,165 INFO [PEWorker-4] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-02-08 08:55:08,167 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 182 msec 2023-02-08 08:55:08,293 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:08,293 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:08,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-08 08:55:08,294 INFO [Listener at localhost/40763] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-02-08 08:55:08,295 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:08,295 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:08,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-02-08 08:55:08,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-02-08 08:55:08,305 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-08 08:55:08,307 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-08 08:55:08,307 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:08,307 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:08,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-02-08 08:55:08,309 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a 2023-02-08 08:55:08,311 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1, FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/recovered.edits] 2023-02-08 08:55:08,314 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/29ed9c3f4fe34fc9821e849e2b850777 2023-02-08 08:55:08,315 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/4134d6741eeb44f6aa6f8f157ae481d1 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/f1/4134d6741eeb44f6aa6f8f157ae481d1 2023-02-08 08:55:08,317 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/recovered.edits/15.seqid to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a/recovered.edits/15.seqid 2023-02-08 08:55:08,317 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testGetAfterCompaction/23f89a76708cf9c5d819f0df80d3c88a 2023-02-08 08:55:08,317 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-02-08 08:55:08,318 DEBUG [PEWorker-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-02-08 08:55:08,318 DEBUG [PEWorker-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-02-08 08:55:08,321 DEBUG [PEWorker-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230208e1f9719dced3498d984698b4a2b134bf to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230208e1f9719dced3498d984698b4a2b134bf 2023-02-08 08:55:08,321 DEBUG [PEWorker-1] backup.HFileArchiver(596): Deleted hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-02-08 08:55:08,322 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-08 08:55:08,343 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-02-08 08:55:08,345 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-02-08 08:55:08,346 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-08 08:55:08,346 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-02-08 08:55:08,346 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846508346"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:08,348 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-08 08:55:08,348 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 23f89a76708cf9c5d819f0df80d3c88a, NAME => 'testGetAfterCompaction,,1675846503107.23f89a76708cf9c5d819f0df80d3c88a.', STARTKEY => '', ENDKEY => ''}] 2023-02-08 08:55:08,348 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-02-08 08:55:08,348 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1675846508348"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:08,349 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-02-08 08:55:08,353 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-08 08:55:08,354 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 54 msec 2023-02-08 08:55:08,408 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:08,408 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:08,409 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-02-08 08:55:08,409 INFO [Listener at localhost/40763] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-02-08 08:55:08,422 INFO [Listener at localhost/40763] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=236 (was 251), OpenFileDescriptor=601 (was 645), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=193 (was 210), ProcessCount=168 (was 168), AvailableMemoryMB=4151 (was 4169) 2023-02-08 08:55:08,432 INFO [Listener at localhost/40763] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=236, OpenFileDescriptor=601, MaxFileDescriptor=60000, SystemLoadAverage=193, ProcessCount=168, AvailableMemoryMB=4150 2023-02-08 08:55:08,432 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:08,432 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:08,433 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-08 08:55:08,434 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-02-08 08:55:08,435 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:55:08,435 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-02-08 08:55:08,436 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:08,436 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:08,436 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:55:08,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-08 08:55:08,438 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:55:08,438 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:55:08,438 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:55:08,438 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410 empty. 2023-02-08 08:55:08,438 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade empty. 2023-02-08 08:55:08,438 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a empty. 2023-02-08 08:55:08,439 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:55:08,439 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:55:08,439 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:55:08,439 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-02-08 08:55:08,537 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:08,537 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:08,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-08 08:55:08,738 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:08,738 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:08,738 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-08 08:55:08,848 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-02-08 08:55:08,850 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7671): creating {ENCODED => 665fcb26db73b9d2ce507202c21c1ade, NAME => 'testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp 2023-02-08 08:55:08,850 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7671): creating {ENCODED => 5f0bc0fd5600fb2bfd54b18349c52a4a, NAME => 'testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp 2023-02-08 08:55:08,850 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7671): creating {ENCODED => 79e77e74725bcdadb1e041ea443ff410, NAME => 'testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp 2023-02-08 08:55:09,039 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:09,039 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:09,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-08 08:55:09,257 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:09,257 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1603): Closing 665fcb26db73b9d2ce507202c21c1ade, disabling compactions & flushes 2023-02-08 08:55:09,257 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:09,257 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:09,257 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. after waiting 0 ms 2023-02-08 08:55:09,257 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:09,257 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:09,257 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1557): Region close journal for 665fcb26db73b9d2ce507202c21c1ade: Waiting for close lock at 1675846509257Disabling compacts and flushes for region at 1675846509257Disabling writes for close at 1675846509257Writing region close event to WAL at 1675846509257Closed at 1675846509257 2023-02-08 08:55:09,261 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:09,261 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:09,261 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1603): Closing 79e77e74725bcdadb1e041ea443ff410, disabling compactions & flushes 2023-02-08 08:55:09,261 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1603): Closing 5f0bc0fd5600fb2bfd54b18349c52a4a, disabling compactions & flushes 2023-02-08 08:55:09,261 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:09,261 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:09,261 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. after waiting 0 ms 2023-02-08 08:55:09,261 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:09,261 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:09,261 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:09,261 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. after waiting 0 ms 2023-02-08 08:55:09,261 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:09,261 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:09,261 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:09,261 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1557): Region close journal for 79e77e74725bcdadb1e041ea443ff410: Waiting for close lock at 1675846509261Disabling compacts and flushes for region at 1675846509261Disabling writes for close at 1675846509261Writing region close event to WAL at 1675846509261Closed at 1675846509261 2023-02-08 08:55:09,261 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1557): Region close journal for 5f0bc0fd5600fb2bfd54b18349c52a4a: Waiting for close lock at 1675846509261Disabling compacts and flushes for region at 1675846509261Disabling writes for close at 1675846509261Writing region close event to WAL at 1675846509261Closed at 1675846509261 2023-02-08 08:55:09,263 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:55:09,264 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675846509264"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846509264"}]},"ts":"1675846509264"} 2023-02-08 08:55:09,264 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675846509264"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846509264"}]},"ts":"1675846509264"} 2023-02-08 08:55:09,264 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675846509264"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846509264"}]},"ts":"1675846509264"} 2023-02-08 08:55:09,266 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-08 08:55:09,267 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:55:09,267 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846509267"}]},"ts":"1675846509267"} 2023-02-08 08:55:09,268 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-02-08 08:55:09,272 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=79e77e74725bcdadb1e041ea443ff410, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5f0bc0fd5600fb2bfd54b18349c52a4a, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=665fcb26db73b9d2ce507202c21c1ade, ASSIGN}] 2023-02-08 08:55:09,273 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5f0bc0fd5600fb2bfd54b18349c52a4a, ASSIGN 2023-02-08 08:55:09,273 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=665fcb26db73b9d2ce507202c21c1ade, ASSIGN 2023-02-08 08:55:09,273 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=79e77e74725bcdadb1e041ea443ff410, ASSIGN 2023-02-08 08:55:09,274 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=665fcb26db73b9d2ce507202c21c1ade, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=false 2023-02-08 08:55:09,274 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=79e77e74725bcdadb1e041ea443ff410, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=false 2023-02-08 08:55:09,274 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5f0bc0fd5600fb2bfd54b18349c52a4a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=false 2023-02-08 08:55:09,425 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=5f0bc0fd5600fb2bfd54b18349c52a4a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:09,425 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=665fcb26db73b9d2ce507202c21c1ade, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:09,425 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=79e77e74725bcdadb1e041ea443ff410, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:09,425 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675846509425"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846509425"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846509425"}]},"ts":"1675846509425"} 2023-02-08 08:55:09,425 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675846509425"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846509425"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846509425"}]},"ts":"1675846509425"} 2023-02-08 08:55:09,425 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675846509425"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846509425"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846509425"}]},"ts":"1675846509425"} 2023-02-08 08:55:09,427 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=81, state=RUNNABLE; OpenRegionProcedure 5f0bc0fd5600fb2bfd54b18349c52a4a, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:55:09,427 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=82, state=RUNNABLE; OpenRegionProcedure 665fcb26db73b9d2ce507202c21c1ade, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:55:09,429 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=80, state=RUNNABLE; OpenRegionProcedure 79e77e74725bcdadb1e041ea443ff410, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:55:09,540 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:09,540 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:09,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-08 08:55:09,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:09,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 665fcb26db73b9d2ce507202c21c1ade, NAME => 'testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade.', STARTKEY => '2', ENDKEY => ''} 2023-02-08 08:55:09,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:55:09,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:09,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:55:09,582 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:55:09,583 INFO [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:55:09,584 DEBUG [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1 2023-02-08 08:55:09,584 DEBUG [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1 2023-02-08 08:55:09,585 INFO [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 665fcb26db73b9d2ce507202c21c1ade columnFamilyName family1 2023-02-08 08:55:09,586 INFO [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] regionserver.HStore(310): Store=665fcb26db73b9d2ce507202c21c1ade/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:09,586 INFO [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:55:09,587 DEBUG [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2 2023-02-08 08:55:09,587 DEBUG [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2 2023-02-08 08:55:09,587 INFO [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 665fcb26db73b9d2ce507202c21c1ade columnFamilyName family2 2023-02-08 08:55:09,589 INFO [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] regionserver.HStore(310): Store=665fcb26db73b9d2ce507202c21c1ade/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:09,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:55:09,590 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:55:09,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:55:09,592 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:55:09,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:55:09,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 665fcb26db73b9d2ce507202c21c1ade; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59533174, jitterRate=-0.11288657784461975}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:55:09,595 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 665fcb26db73b9d2ce507202c21c1ade: Running coprocessor pre-open hook at 1675846509582Writing region info on filesystem at 1675846509582Initializing all the Stores at 1675846509583 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846509583Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846509583Cleaning up temporary data from old regions at 1675846509590 (+7 ms)Cleaning up detritus from prior splits at 1675846509591 (+1 ms)Running coprocessor post-open hooks at 1675846509595 (+4 ms)Region opened successfully at 1675846509595 2023-02-08 08:55:09,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade., pid=84, masterSystemTime=1675846509579 2023-02-08 08:55:09,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:09,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:09,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:09,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 79e77e74725bcdadb1e041ea443ff410, NAME => 'testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410.', STARTKEY => '', ENDKEY => '1'} 2023-02-08 08:55:09,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:55:09,597 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=665fcb26db73b9d2ce507202c21c1ade, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:09,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:09,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:55:09,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:55:09,597 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675846509597"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846509597"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846509597"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846509597"}]},"ts":"1675846509597"} 2023-02-08 08:55:09,599 INFO [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:55:09,600 DEBUG [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1 2023-02-08 08:55:09,600 DEBUG [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1 2023-02-08 08:55:09,600 INFO [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 79e77e74725bcdadb1e041ea443ff410 columnFamilyName family1 2023-02-08 08:55:09,600 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=82 2023-02-08 08:55:09,600 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=82, state=SUCCESS; OpenRegionProcedure 665fcb26db73b9d2ce507202c21c1ade, server=jenkins-hbase4.apache.org,42239,1675846413348 in 172 msec 2023-02-08 08:55:09,601 INFO [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] regionserver.HStore(310): Store=79e77e74725bcdadb1e041ea443ff410/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:09,601 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=665fcb26db73b9d2ce507202c21c1ade, ASSIGN in 329 msec 2023-02-08 08:55:09,601 INFO [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:55:09,602 DEBUG [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2 2023-02-08 08:55:09,602 DEBUG [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2 2023-02-08 08:55:09,603 INFO [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 79e77e74725bcdadb1e041ea443ff410 columnFamilyName family2 2023-02-08 08:55:09,604 INFO [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] regionserver.HStore(310): Store=79e77e74725bcdadb1e041ea443ff410/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:09,605 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:55:09,605 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:55:09,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:55:09,607 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:55:09,609 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:55:09,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 79e77e74725bcdadb1e041ea443ff410; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64801064, jitterRate=-0.03438889980316162}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:55:09,610 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 79e77e74725bcdadb1e041ea443ff410: Running coprocessor pre-open hook at 1675846509598Writing region info on filesystem at 1675846509598Initializing all the Stores at 1675846509598Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846509598Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846509598Cleaning up temporary data from old regions at 1675846509606 (+8 ms)Cleaning up detritus from prior splits at 1675846509606Running coprocessor post-open hooks at 1675846509609 (+3 ms)Region opened successfully at 1675846509609 2023-02-08 08:55:09,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410., pid=85, masterSystemTime=1675846509579 2023-02-08 08:55:09,611 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:09,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:09,611 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:09,611 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5f0bc0fd5600fb2bfd54b18349c52a4a, NAME => 'testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a.', STARTKEY => '1', ENDKEY => '2'} 2023-02-08 08:55:09,611 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=79e77e74725bcdadb1e041ea443ff410, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:09,612 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675846509611"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846509611"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846509611"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846509611"}]},"ts":"1675846509611"} 2023-02-08 08:55:09,612 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:55:09,612 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:09,612 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:55:09,612 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:55:09,613 INFO [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:55:09,614 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=80 2023-02-08 08:55:09,614 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=80, state=SUCCESS; OpenRegionProcedure 79e77e74725bcdadb1e041ea443ff410, server=jenkins-hbase4.apache.org,42239,1675846413348 in 185 msec 2023-02-08 08:55:09,614 DEBUG [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1 2023-02-08 08:55:09,614 DEBUG [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1 2023-02-08 08:55:09,615 INFO [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5f0bc0fd5600fb2bfd54b18349c52a4a columnFamilyName family1 2023-02-08 08:55:09,615 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=79e77e74725bcdadb1e041ea443ff410, ASSIGN in 344 msec 2023-02-08 08:55:09,616 INFO [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] regionserver.HStore(310): Store=5f0bc0fd5600fb2bfd54b18349c52a4a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:09,616 INFO [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:55:09,617 DEBUG [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2 2023-02-08 08:55:09,617 DEBUG [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2 2023-02-08 08:55:09,617 INFO [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5f0bc0fd5600fb2bfd54b18349c52a4a columnFamilyName family2 2023-02-08 08:55:09,618 INFO [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] regionserver.HStore(310): Store=5f0bc0fd5600fb2bfd54b18349c52a4a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:09,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:55:09,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:55:09,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:55:09,621 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:55:09,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:55:09,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 5f0bc0fd5600fb2bfd54b18349c52a4a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67721077, jitterRate=0.009122684597969055}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:55:09,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 5f0bc0fd5600fb2bfd54b18349c52a4a: Running coprocessor pre-open hook at 1675846509612Writing region info on filesystem at 1675846509612Initializing all the Stores at 1675846509613 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846509613Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846509613Cleaning up temporary data from old regions at 1675846509620 (+7 ms)Cleaning up detritus from prior splits at 1675846509620Running coprocessor post-open hooks at 1675846509623 (+3 ms)Region opened successfully at 1675846509623 2023-02-08 08:55:09,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a., pid=83, masterSystemTime=1675846509579 2023-02-08 08:55:09,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:09,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:09,625 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=5f0bc0fd5600fb2bfd54b18349c52a4a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:09,625 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675846509625"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846509625"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846509625"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846509625"}]},"ts":"1675846509625"} 2023-02-08 08:55:09,628 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=81 2023-02-08 08:55:09,628 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=81, state=SUCCESS; OpenRegionProcedure 5f0bc0fd5600fb2bfd54b18349c52a4a, server=jenkins-hbase4.apache.org,42239,1675846413348 in 200 msec 2023-02-08 08:55:09,630 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-02-08 08:55:09,630 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5f0bc0fd5600fb2bfd54b18349c52a4a, ASSIGN in 358 msec 2023-02-08 08:55:09,631 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:55:09,631 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846509631"}]},"ts":"1675846509631"} 2023-02-08 08:55:09,632 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-02-08 08:55:09,634 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:55:09,635 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.2000 sec 2023-02-08 08:55:10,500 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-02-08 08:55:10,541 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:10,541 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:10,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-08 08:55:10,542 INFO [Listener at localhost/40763] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-02-08 08:55:10,548 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:10,549 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:10,550 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:10,552 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:10,553 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:10,553 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:10,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-08 08:55:10,555 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-08 08:55:10,555 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:10,555 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-08 08:55:10,555 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-08 08:55:10,555 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:10,555 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:10,557 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:10,558 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:10,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:10,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:10,558 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:10,558 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:10,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:10,558 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:10,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:10,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:10,558 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-08 08:55:10,559 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-08 08:55:10,559 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-08 08:55:10,559 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:10,559 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-08 08:55:10,559 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:10,559 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 08:55:10,559 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:10,559 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:10,559 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. started... 2023-02-08 08:55:10,559 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. started... 2023-02-08 08:55:10,559 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:10,560 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. started... 2023-02-08 08:55:10,560 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 79e77e74725bcdadb1e041ea443ff410 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-08 08:55:10,560 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 665fcb26db73b9d2ce507202c21c1ade 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-08 08:55:10,560 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 5f0bc0fd5600fb2bfd54b18349c52a4a 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-08 08:55:10,572 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208e31e783a362b4ccdb2c77b63c75f96cc to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208e31e783a362b4ccdb2c77b63c75f96cc 2023-02-08 08:55:10,974 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302082721e52113d24baca886c5b893723eee to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302082721e52113d24baca886c5b893723eee 2023-02-08 08:55:10,974 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302088573e6ea6b2449f2b300daa6fa8fb08d to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302088573e6ea6b2449f2b300daa6fa8fb08d 2023-02-08 08:55:10,978 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family1/284ce0dd25ac49459e03d8d9b0666ef9 2023-02-08 08:55:10,978 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family1/7c18a1b1124b437aa6ac4ca4574bd6aa 2023-02-08 08:55:11,379 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family1/0188e80890ce408c90da25548b3d2a57 2023-02-08 08:55:11,391 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208a67f884cad3d40238bc523a59405b027 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208a67f884cad3d40238bc523a59405b027 2023-02-08 08:55:11,391 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302082c30bcfc25054141a3239a271f481236 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302082c30bcfc25054141a3239a271f481236 2023-02-08 08:55:11,793 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208909c1d78aa524f7bb29ed96ee3bd693c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208909c1d78aa524f7bb29ed96ee3bd693c 2023-02-08 08:55:11,795 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family2/8eee8a31f2ca450389d1a3ee3f1350b0 2023-02-08 08:55:11,796 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family2/729ebf7c577943268499efb8cb0a07ba 2023-02-08 08:55:11,798 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family2/6980bf1172c44a8c8fb61eef66ebcc3e 2023-02-08 08:55:11,800 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family1/7c18a1b1124b437aa6ac4ca4574bd6aa as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/7c18a1b1124b437aa6ac4ca4574bd6aa 2023-02-08 08:55:11,800 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family1/284ce0dd25ac49459e03d8d9b0666ef9 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/284ce0dd25ac49459e03d8d9b0666ef9 2023-02-08 08:55:11,804 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family1/0188e80890ce408c90da25548b3d2a57 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/0188e80890ce408c90da25548b3d2a57 2023-02-08 08:55:11,806 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/7c18a1b1124b437aa6ac4ca4574bd6aa, entries=4, sequenceid=5, filesize=5.4 K 2023-02-08 08:55:11,806 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/284ce0dd25ac49459e03d8d9b0666ef9, entries=4, sequenceid=5, filesize=5.4 K 2023-02-08 08:55:11,807 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family2/729ebf7c577943268499efb8cb0a07ba as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/729ebf7c577943268499efb8cb0a07ba 2023-02-08 08:55:11,807 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family2/8eee8a31f2ca450389d1a3ee3f1350b0 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/8eee8a31f2ca450389d1a3ee3f1350b0 2023-02-08 08:55:11,811 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/0188e80890ce408c90da25548b3d2a57, entries=4, sequenceid=5, filesize=5.4 K 2023-02-08 08:55:11,812 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family2/6980bf1172c44a8c8fb61eef66ebcc3e as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/6980bf1172c44a8c8fb61eef66ebcc3e 2023-02-08 08:55:11,812 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/729ebf7c577943268499efb8cb0a07ba, entries=2, sequenceid=5, filesize=5.1 K 2023-02-08 08:55:11,812 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/8eee8a31f2ca450389d1a3ee3f1350b0, entries=2, sequenceid=5, filesize=5.1 K 2023-02-08 08:55:11,813 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 665fcb26db73b9d2ce507202c21c1ade in 1253ms, sequenceid=5, compaction requested=false 2023-02-08 08:55:11,813 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 5f0bc0fd5600fb2bfd54b18349c52a4a in 1253ms, sequenceid=5, compaction requested=false 2023-02-08 08:55:11,813 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 665fcb26db73b9d2ce507202c21c1ade: 2023-02-08 08:55:11,814 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 5f0bc0fd5600fb2bfd54b18349c52a4a: 2023-02-08 08:55:11,814 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:11,814 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:11,814 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 08:55:11,814 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 08:55:11,817 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/6980bf1172c44a8c8fb61eef66ebcc3e, entries=2, sequenceid=5, filesize=5.1 K 2023-02-08 08:55:11,817 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 79e77e74725bcdadb1e041ea443ff410 in 1257ms, sequenceid=5, compaction requested=false 2023-02-08 08:55:11,817 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 79e77e74725bcdadb1e041ea443ff410: 2023-02-08 08:55:11,817 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:11,817 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 08:55:11,817 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 08:55:11,817 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:11,818 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-08 08:55:11,818 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-08 08:55:11,819 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:11,819 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:11,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:11,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:11,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:11,819 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:11,819 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:11,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:11,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:11,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:11,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:11,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:11,821 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-08 08:55:11,821 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2195cc03[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:11,821 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-08 08:55:11,821 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:11,822 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:11,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:11,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:11,822 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-08 08:55:11,822 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-08 08:55:11,822 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:11,822 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:11,822 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:55:11,823 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-08 08:55:11,823 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:11,823 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:11,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:11,823 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-08 08:55:11,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:11,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:11,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:11,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:11,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:11,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:11,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:11,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:11,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:11,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:55:11,825 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-08 08:55:11,825 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-08 08:55:11,825 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:11,825 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:11,825 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-08 08:55:11,825 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:11,830 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:11,830 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:11,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:11,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:11,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:11,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:11,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:11,830 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:11,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:11,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:11,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:11,830 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:11,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:11,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:11,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:11,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:11,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:11,831 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:11,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:11,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:11,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:11,833 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:11,833 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:11,833 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:11,833 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:11,833 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-08 08:55:11,833 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:11,833 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:11,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:11,833 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:11,833 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:11,833 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:11,833 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:11,833 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:11,833 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-08 08:55:11,834 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:11,833 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:11,834 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:11,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:11,934 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:11,934 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:11,934 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:11,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:11,935 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:11,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:11,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:11,938 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:11,938 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:11,938 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:11,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:11,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-08 08:55:11,940 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-08 08:55:11,940 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:11,940 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-08 08:55:11,940 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-08 08:55:11,941 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:11,941 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:11,942 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:11,942 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:11,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:11,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:11,942 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:11,942 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:11,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:11,943 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:11,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:11,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:11,943 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-08 08:55:11,943 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-08 08:55:11,943 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-08 08:55:11,943 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-08 08:55:11,943 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:11,943 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-08 08:55:11,944 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:11,944 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. started... 2023-02-08 08:55:11,944 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:11,944 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:11,944 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 665fcb26db73b9d2ce507202c21c1ade 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-08 08:55:11,944 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 08:55:11,944 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:11,944 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. started... 2023-02-08 08:55:11,944 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. started... 2023-02-08 08:55:11,944 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 79e77e74725bcdadb1e041ea443ff410 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-08 08:55:11,944 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 5f0bc0fd5600fb2bfd54b18349c52a4a 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-08 08:55:12,358 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208547dd82594f94b97b01fec7e73f90b7e to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208547dd82594f94b97b01fec7e73f90b7e 2023-02-08 08:55:12,358 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302085696d16ea6fb44fe8532f61a0ed98c5c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302085696d16ea6fb44fe8532f61a0ed98c5c 2023-02-08 08:55:12,358 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208ee9b7e1c09a04572bfad5423c41f1272 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208ee9b7e1c09a04572bfad5423c41f1272 2023-02-08 08:55:12,763 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family1/573bbd30fdb14700861a01a224793061 2023-02-08 08:55:12,763 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family1/5477f14eaa574e989fe111de5de2a17f 2023-02-08 08:55:12,763 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family1/acb7ff0c34d84d7695b97d48daefdb93 2023-02-08 08:55:13,177 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020847397442cb504fcf889973f84a22c5f9 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020847397442cb504fcf889973f84a22c5f9 2023-02-08 08:55:13,177 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020854c51eb8b80e424697b30e0801b27a40 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020854c51eb8b80e424697b30e0801b27a40 2023-02-08 08:55:13,179 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208109b8e65fbf749c88b76ee104ef9f46e to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208109b8e65fbf749c88b76ee104ef9f46e 2023-02-08 08:55:13,182 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family2/e5a1c334760a448d8a1d5fa71666c54c 2023-02-08 08:55:13,184 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family2/dc241889a15f41ffbbbba991bb167763 2023-02-08 08:55:13,187 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family1/573bbd30fdb14700861a01a224793061 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/573bbd30fdb14700861a01a224793061 2023-02-08 08:55:13,188 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family1/5477f14eaa574e989fe111de5de2a17f as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/5477f14eaa574e989fe111de5de2a17f 2023-02-08 08:55:13,190 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/573bbd30fdb14700861a01a224793061, entries=4, sequenceid=9, filesize=5.4 K 2023-02-08 08:55:13,191 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family2/e5a1c334760a448d8a1d5fa71666c54c as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/e5a1c334760a448d8a1d5fa71666c54c 2023-02-08 08:55:13,191 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/5477f14eaa574e989fe111de5de2a17f, entries=4, sequenceid=9, filesize=5.4 K 2023-02-08 08:55:13,192 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family2/dc241889a15f41ffbbbba991bb167763 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/dc241889a15f41ffbbbba991bb167763 2023-02-08 08:55:13,195 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/e5a1c334760a448d8a1d5fa71666c54c, entries=2, sequenceid=9, filesize=5.1 K 2023-02-08 08:55:13,195 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/dc241889a15f41ffbbbba991bb167763, entries=2, sequenceid=9, filesize=5.1 K 2023-02-08 08:55:13,195 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 665fcb26db73b9d2ce507202c21c1ade in 1251ms, sequenceid=9, compaction requested=false 2023-02-08 08:55:13,196 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 665fcb26db73b9d2ce507202c21c1ade: 2023-02-08 08:55:13,196 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:13,196 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 08:55:13,196 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 5f0bc0fd5600fb2bfd54b18349c52a4a in 1252ms, sequenceid=9, compaction requested=false 2023-02-08 08:55:13,196 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 5f0bc0fd5600fb2bfd54b18349c52a4a: 2023-02-08 08:55:13,196 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:13,196 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 08:55:13,583 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family2/60be5913ca21413f86163127d6eed588 2023-02-08 08:55:13,587 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family1/acb7ff0c34d84d7695b97d48daefdb93 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/acb7ff0c34d84d7695b97d48daefdb93 2023-02-08 08:55:13,590 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/acb7ff0c34d84d7695b97d48daefdb93, entries=4, sequenceid=9, filesize=5.4 K 2023-02-08 08:55:13,591 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family2/60be5913ca21413f86163127d6eed588 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/60be5913ca21413f86163127d6eed588 2023-02-08 08:55:13,594 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/60be5913ca21413f86163127d6eed588, entries=2, sequenceid=9, filesize=5.1 K 2023-02-08 08:55:13,594 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 79e77e74725bcdadb1e041ea443ff410 in 1650ms, sequenceid=9, compaction requested=false 2023-02-08 08:55:13,595 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 79e77e74725bcdadb1e041ea443ff410: 2023-02-08 08:55:13,595 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:13,595 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 08:55:13,595 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 08:55:13,595 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:13,595 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-08 08:55:13,595 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-08 08:55:13,596 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:13,596 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:13,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:13,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:13,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:13,597 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:13,597 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:13,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:13,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:13,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:13,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:13,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:13,598 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-08 08:55:13,598 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@163b1b13[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:13,598 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-08 08:55:13,598 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:13,600 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:13,600 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:13,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:13,600 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-08 08:55:13,600 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-08 08:55:13,600 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:55:13,600 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:13,600 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:13,601 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-08 08:55:13,601 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:13,601 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:13,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:13,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:13,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:13,601 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-08 08:55:13,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:13,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:13,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:13,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:13,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:13,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:13,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:13,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:55:13,603 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-08 08:55:13,603 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-08 08:55:13,603 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:13,603 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:13,603 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-08 08:55:13,603 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:13,604 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:13,604 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:13,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:13,604 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:13,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:13,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:13,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:13,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:13,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:13,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:13,605 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:13,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:13,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:13,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:13,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:13,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:13,606 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:13,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:13,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:13,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:13,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:13,608 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:13,608 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:13,608 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:13,608 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:13,608 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:13,608 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:13,608 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-08 08:55:13,608 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:13,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:13,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:13,608 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:13,608 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:13,608 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:13,608 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:13,608 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-08 08:55:13,608 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:13,608 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-08 08:55:13,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:13,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:13,659 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 08:55:13,708 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:13,709 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:13,709 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:13,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:13,710 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:13,711 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:13,711 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:13,712 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:13,713 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:13,713 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:13,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:13,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-08 08:55:13,715 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-08 08:55:13,715 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:13,715 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-08 08:55:13,715 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-08 08:55:13,715 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:13,715 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:13,717 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:13,717 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:13,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:13,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:13,717 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:13,717 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:13,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:13,718 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:13,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:13,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:13,718 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-08 08:55:13,718 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-08 08:55:13,718 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-08 08:55:13,718 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-08 08:55:13,718 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:13,718 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-08 08:55:13,719 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:13,719 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. started... 2023-02-08 08:55:13,719 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:13,719 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 08:55:13,719 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:13,719 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:13,719 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. started... 2023-02-08 08:55:13,719 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. started... 2023-02-08 08:55:13,719 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 665fcb26db73b9d2ce507202c21c1ade 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-08 08:55:13,719 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 5f0bc0fd5600fb2bfd54b18349c52a4a 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-08 08:55:13,719 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 79e77e74725bcdadb1e041ea443ff410 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-08 08:55:13,746 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208df405c3a445f4c1cad9d8184abd4dd96 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208df405c3a445f4c1cad9d8184abd4dd96 2023-02-08 08:55:13,746 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020842137fd8be5a4f54a55e79a58a213e9a to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020842137fd8be5a4f54a55e79a58a213e9a 2023-02-08 08:55:14,139 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020836ad795446ba45d68401de818d5a02f4 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020836ad795446ba45d68401de818d5a02f4 2023-02-08 08:55:14,150 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family1/7df916e5f1c94fa0a8259f67ef2aec54 2023-02-08 08:55:14,156 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family1/d63b5936006140559ed94f3e9779142c 2023-02-08 08:55:14,169 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302088fe72b3b4a7f48a3a8a02594a7eba7e2 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302088fe72b3b4a7f48a3a8a02594a7eba7e2 2023-02-08 08:55:14,543 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family1/dea8e16f3dd74291809c3038eb6c855c 2023-02-08 08:55:14,564 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020870729d0a550e41bb8659552a71588f5f to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020870729d0a550e41bb8659552a71588f5f 2023-02-08 08:55:14,573 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family2/71e29344e20f4cd38dd724239183b532 2023-02-08 08:55:14,577 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family1/d63b5936006140559ed94f3e9779142c as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/d63b5936006140559ed94f3e9779142c 2023-02-08 08:55:14,580 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/d63b5936006140559ed94f3e9779142c, entries=4, sequenceid=13, filesize=5.4 K 2023-02-08 08:55:14,581 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family2/71e29344e20f4cd38dd724239183b532 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/71e29344e20f4cd38dd724239183b532 2023-02-08 08:55:14,584 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/71e29344e20f4cd38dd724239183b532, entries=2, sequenceid=13, filesize=5.1 K 2023-02-08 08:55:14,584 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 665fcb26db73b9d2ce507202c21c1ade in 865ms, sequenceid=13, compaction requested=true 2023-02-08 08:55:14,584 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 665fcb26db73b9d2ce507202c21c1ade: 2023-02-08 08:55:14,584 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:14,585 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 08:55:14,957 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020820354c0d3bc849f09a31eec434c6b35d to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020820354c0d3bc849f09a31eec434c6b35d 2023-02-08 08:55:14,968 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family2/a50110e6550e41a68e419e663403be7b 2023-02-08 08:55:14,972 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family1/7df916e5f1c94fa0a8259f67ef2aec54 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/7df916e5f1c94fa0a8259f67ef2aec54 2023-02-08 08:55:14,975 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/7df916e5f1c94fa0a8259f67ef2aec54, entries=4, sequenceid=13, filesize=5.4 K 2023-02-08 08:55:14,975 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family2/a50110e6550e41a68e419e663403be7b as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/a50110e6550e41a68e419e663403be7b 2023-02-08 08:55:14,978 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/a50110e6550e41a68e419e663403be7b, entries=2, sequenceid=13, filesize=5.1 K 2023-02-08 08:55:14,979 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 79e77e74725bcdadb1e041ea443ff410 in 1260ms, sequenceid=13, compaction requested=true 2023-02-08 08:55:14,979 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 79e77e74725bcdadb1e041ea443ff410: 2023-02-08 08:55:14,979 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:14,979 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 08:55:15,361 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family2/eab5fe674d46495898c58cd9b8ca04bc 2023-02-08 08:55:15,367 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family1/dea8e16f3dd74291809c3038eb6c855c as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/dea8e16f3dd74291809c3038eb6c855c 2023-02-08 08:55:15,371 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/dea8e16f3dd74291809c3038eb6c855c, entries=4, sequenceid=13, filesize=5.4 K 2023-02-08 08:55:15,372 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family2/eab5fe674d46495898c58cd9b8ca04bc as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/eab5fe674d46495898c58cd9b8ca04bc 2023-02-08 08:55:15,377 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/eab5fe674d46495898c58cd9b8ca04bc, entries=2, sequenceid=13, filesize=5.1 K 2023-02-08 08:55:15,377 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 5f0bc0fd5600fb2bfd54b18349c52a4a in 1658ms, sequenceid=13, compaction requested=true 2023-02-08 08:55:15,377 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 5f0bc0fd5600fb2bfd54b18349c52a4a: 2023-02-08 08:55:15,377 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:15,377 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 08:55:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 08:55:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-08 08:55:15,378 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-08 08:55:15,379 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:15,379 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:15,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:15,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:15,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:15,380 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:15,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:15,380 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:15,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:15,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:15,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:15,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:15,381 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-08 08:55:15,381 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@11b04e32[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:15,381 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-08 08:55:15,381 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:15,382 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:15,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:15,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:15,382 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-08 08:55:15,382 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-08 08:55:15,382 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:15,382 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:15,382 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:55:15,392 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-08 08:55:15,392 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:15,392 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:15,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:15,392 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-08 08:55:15,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:15,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:15,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:15,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:15,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:15,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:15,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:15,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:15,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:15,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:55:15,394 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-08 08:55:15,394 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-08 08:55:15,394 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:15,394 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:15,394 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-08 08:55:15,394 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:15,395 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:15,395 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:15,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:15,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:15,395 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:15,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:15,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:15,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:15,396 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:15,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:15,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:15,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:15,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:15,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:15,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:15,397 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:15,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:15,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:15,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:15,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:15,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:15,398 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:15,398 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:15,398 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:15,399 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:15,399 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-08 08:55:15,399 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:15,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:15,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:15,399 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:15,399 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:15,399 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-08 08:55:15,399 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-08 08:55:15,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:15,399 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:15,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:15,399 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:15,399 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:15,399 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:15,399 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:15,499 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:15,499 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:15,499 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:15,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:15,501 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:15,501 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:15,502 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:15,504 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:15,504 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:15,504 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:15,506 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:15,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-08 08:55:15,507 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-08 08:55:15,507 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:15,507 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-08 08:55:15,507 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-08 08:55:15,507 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:15,507 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:15,508 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:15,508 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:15,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:15,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:15,509 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:15,509 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:15,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:15,509 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:15,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:15,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:15,510 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-08 08:55:15,510 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-08 08:55:15,510 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-08 08:55:15,510 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-08 08:55:15,510 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:15,510 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-08 08:55:15,511 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:15,511 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. started... 2023-02-08 08:55:15,511 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 665fcb26db73b9d2ce507202c21c1ade 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-08 08:55:15,511 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:15,511 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. started... 2023-02-08 08:55:15,511 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 5f0bc0fd5600fb2bfd54b18349c52a4a 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-08 08:55:15,512 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 08:55:15,512 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:15,512 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:15,512 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. started... 2023-02-08 08:55:15,512 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 79e77e74725bcdadb1e041ea443ff410 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-08 08:55:15,551 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208f5ce04e42d824d54a5cf228d419280d6 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208f5ce04e42d824d54a5cf228d419280d6 2023-02-08 08:55:15,555 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family1/dc99f56e914e4cddbaa43f5992b35a20 2023-02-08 08:55:15,943 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208845abc6337634d249733ad7bc14255fd to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230208845abc6337634d249733ad7bc14255fd 2023-02-08 08:55:15,943 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208e0a819e1913c40aebbd0261f70920513 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208e0a819e1913c40aebbd0261f70920513 2023-02-08 08:55:15,967 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302080420b0823ab2472fbd59ab7c177a0363 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302080420b0823ab2472fbd59ab7c177a0363 2023-02-08 08:55:16,349 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family1/3a17e09e335446e1b6ed1f591fb25d82 2023-02-08 08:55:16,353 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family1/55af94394e4d4776a9a83012121fb840 2023-02-08 08:55:16,372 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family2/d7411eeea5ed4ed2936ab18ceb564590 2023-02-08 08:55:16,376 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family1/dc99f56e914e4cddbaa43f5992b35a20 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/dc99f56e914e4cddbaa43f5992b35a20 2023-02-08 08:55:16,379 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/dc99f56e914e4cddbaa43f5992b35a20, entries=4, sequenceid=17, filesize=5.4 K 2023-02-08 08:55:16,379 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family2/d7411eeea5ed4ed2936ab18ceb564590 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/d7411eeea5ed4ed2936ab18ceb564590 2023-02-08 08:55:16,382 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/d7411eeea5ed4ed2936ab18ceb564590, entries=2, sequenceid=17, filesize=5.1 K 2023-02-08 08:55:16,383 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 79e77e74725bcdadb1e041ea443ff410 in 871ms, sequenceid=17, compaction requested=true 2023-02-08 08:55:16,383 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 79e77e74725bcdadb1e041ea443ff410: 2023-02-08 08:55:16,383 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:16,383 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 08:55:16,763 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302086f950b436eda41f0962a5eccf190e571 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302086f950b436eda41f0962a5eccf190e571 2023-02-08 08:55:16,766 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302086a575d9587e24ca2aadc4147fd222235 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302086a575d9587e24ca2aadc4147fd222235 2023-02-08 08:55:16,768 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family2/299eac1ba6e746f28065bcf304ba36fd 2023-02-08 08:55:16,771 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family1/3a17e09e335446e1b6ed1f591fb25d82 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/3a17e09e335446e1b6ed1f591fb25d82 2023-02-08 08:55:16,774 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/3a17e09e335446e1b6ed1f591fb25d82, entries=4, sequenceid=17, filesize=5.4 K 2023-02-08 08:55:16,775 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family2/299eac1ba6e746f28065bcf304ba36fd as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/299eac1ba6e746f28065bcf304ba36fd 2023-02-08 08:55:16,778 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/299eac1ba6e746f28065bcf304ba36fd, entries=2, sequenceid=17, filesize=5.1 K 2023-02-08 08:55:16,779 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 665fcb26db73b9d2ce507202c21c1ade in 1268ms, sequenceid=17, compaction requested=true 2023-02-08 08:55:16,779 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 665fcb26db73b9d2ce507202c21c1ade: 2023-02-08 08:55:16,779 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:16,779 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 08:55:17,170 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family2/12f2df8055fc4e4da85427eb05d0bbff 2023-02-08 08:55:17,174 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family1/55af94394e4d4776a9a83012121fb840 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/55af94394e4d4776a9a83012121fb840 2023-02-08 08:55:17,177 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/55af94394e4d4776a9a83012121fb840, entries=4, sequenceid=17, filesize=5.4 K 2023-02-08 08:55:17,178 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family2/12f2df8055fc4e4da85427eb05d0bbff as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/12f2df8055fc4e4da85427eb05d0bbff 2023-02-08 08:55:17,182 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/12f2df8055fc4e4da85427eb05d0bbff, entries=2, sequenceid=17, filesize=5.1 K 2023-02-08 08:55:17,183 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 5f0bc0fd5600fb2bfd54b18349c52a4a in 1672ms, sequenceid=17, compaction requested=true 2023-02-08 08:55:17,183 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 5f0bc0fd5600fb2bfd54b18349c52a4a: 2023-02-08 08:55:17,183 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:17,183 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 08:55:17,183 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 08:55:17,183 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:17,183 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-08 08:55:17,183 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-08 08:55:17,185 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:17,185 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:17,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:17,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:17,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:17,185 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:17,185 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:17,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:17,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:17,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:17,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:17,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:17,186 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-08 08:55:17,186 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@23bcefe6[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:17,186 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-08 08:55:17,186 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:17,187 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:17,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:17,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:17,187 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-08 08:55:17,188 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-08 08:55:17,188 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:17,188 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:17,188 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:55:17,190 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-08 08:55:17,190 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:17,190 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:17,190 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-08 08:55:17,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:17,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:17,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:17,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:17,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:17,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:17,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:17,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:17,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:17,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:17,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:55:17,193 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-08 08:55:17,193 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:17,193 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:17,193 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-08 08:55:17,193 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:17,193 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-08 08:55:17,194 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:17,194 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:17,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:17,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:17,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:17,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:17,194 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:17,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:17,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:17,195 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:17,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:17,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:17,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:17,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:17,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:17,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:17,196 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:17,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:17,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:17,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:17,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:17,198 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:17,198 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:17,198 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:17,198 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:17,198 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:17,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:17,198 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-08 08:55:17,198 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-08 08:55:17,198 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:17,198 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:17,198 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:17,198 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:17,198 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:17,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:17,198 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:17,198 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-08 08:55:17,198 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:17,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:17,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:17,298 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:17,299 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:17,299 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:17,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:17,308 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:17,308 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:17,309 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1675846517299 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-08 08:55:17,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1675846517309 (current time:1675846517309). 2023-02-08 08:55:17,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-02-08 08:55:17,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] zookeeper.ReadOnlyZKClient(139): Connect 0x33a76787 to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:55:17,315 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@43d9689b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:55:17,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:55:17,323 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56922, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:55:17,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x33a76787 to 127.0.0.1:61594 2023-02-08 08:55:17,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:55:17,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-02-08 08:55:17,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1675846517299 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-08 08:55:17,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-08 08:55:17,333 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-08 08:55:17,334 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1675847117334 2023-02-08 08:55:17,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1675846517299 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-08 08:55:17,334 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1675846517299 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-02-08 08:55:17,335 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-08 08:55:17,336 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-08 08:55:17,336 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 3 msec 2023-02-08 08:55:17,336 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-08 08:55:17,337 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1675847117337 2023-02-08 08:55:17,339 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1675846517299 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-02-08 08:55:17,339 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-02-08 08:55:17,439 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 08:55:17,440 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:17,440 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:17,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675846517299 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 08:55:17,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675846517299 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-08 08:55:17,443 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-02-08 08:55:17,643 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 08:55:17,643 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:17,643 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:17,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675846517299 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 08:55:17,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675846517299 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-08 08:55:17,644 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-02-08 08:55:17,748 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1675846517299 2023-02-08 08:55:17,748 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1675846517299' 2023-02-08 08:55:17,748 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-02-08 08:55:17,749 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1675846517299' starting 'acquire' 2023-02-08 08:55:17,749 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1675846517299', kicking off acquire phase on members. 2023-02-08 08:55:17,749 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1675846517299 2023-02-08 08:55:17,749 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1675846517299 2023-02-08 08:55:17,750 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1675846517299/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:17,750 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-02-08 08:55:17,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-02-08 08:55:17,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-08 08:55:17,751 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1675846517299/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:17,751 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:17,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1675846517299 2023-02-08 08:55:17,751 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1675846517299 2023-02-08 08:55:17,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-02-08 08:55:17,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1675846517299 2023-02-08 08:55:17,752 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1675846517299 from table testCompactionWithHFileLink type FLUSH 2023-02-08 08:55:17,754 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1675846517299 2023-02-08 08:55:17,754 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1675846517299' with timeout 300000ms 2023-02-08 08:55:17,754 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-02-08 08:55:17,754 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1675846517299' starting 'acquire' stage 2023-02-08 08:55:17,754 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1675846517299' locally acquired 2023-02-08 08:55:17,754 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (snaptb-1675846517299) in zk 2023-02-08 08:55:17,755 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1675846517299 2023-02-08 08:55:17,755 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1675846517299/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:17,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1675846517299/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:17,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:17,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-08 08:55:17,756 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1675846517299 2023-02-08 08:55:17,756 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1675846517299' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:17,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:17,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:17,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675846517299 2023-02-08 08:55:17,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:17,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:17,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'snaptb-1675846517299' on coordinator 2023-02-08 08:55:17,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2c46ca46[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:17,757 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1675846517299' starting 'in-barrier' execution. 2023-02-08 08:55:17,757 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1675846517299 2023-02-08 08:55:17,758 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675846517299 2023-02-08 08:55:17,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1675846517299 2023-02-08 08:55:17,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1675846517299 2023-02-08 08:55:17,758 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1675846517299' received 'reached' from coordinator. 2023-02-08 08:55:17,758 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1675846517299/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:17,758 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:17,759 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:17,759 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. started... 2023-02-08 08:55:17,759 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-0] regionserver.HRegion(2445): Flush status journal for 665fcb26db73b9d2ce507202c21c1ade: 2023-02-08 08:55:17,760 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:17,760 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. started... 2023-02-08 08:55:17,760 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-1] regionserver.HRegion(2445): Flush status journal for 5f0bc0fd5600fb2bfd54b18349c52a4a: 2023-02-08 08:55:17,760 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade.' region-info for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,760 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a.' region-info for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,762 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-02-08 08:55:17,762 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:17,762 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-02-08 08:55:17,762 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. started... 2023-02-08 08:55:17,762 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-2] regionserver.HRegion(2445): Flush status journal for 79e77e74725bcdadb1e041ea443ff410: 2023-02-08 08:55:17,762 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410.' region-info for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,765 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-08 08:55:17,765 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-08 08:55:17,765 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-08 08:55:17,912 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 08:55:17,912 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-02-08 08:55:17,912 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7ecd2260 to 127.0.0.1:61594 2023-02-08 08:55:17,912 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:55:17,912 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/284ce0dd25ac49459e03d8d9b0666ef9, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/5477f14eaa574e989fe111de5de2a17f, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/dea8e16f3dd74291809c3038eb6c855c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/55af94394e4d4776a9a83012121fb840] hfiles 2023-02-08 08:55:17,912 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/7c18a1b1124b437aa6ac4ca4574bd6aa, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/573bbd30fdb14700861a01a224793061, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/d63b5936006140559ed94f3e9779142c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/3a17e09e335446e1b6ed1f591fb25d82] hfiles 2023-02-08 08:55:17,912 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/0188e80890ce408c90da25548b3d2a57, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/acb7ff0c34d84d7695b97d48daefdb93, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/7df916e5f1c94fa0a8259f67ef2aec54, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/dc99f56e914e4cddbaa43f5992b35a20] hfiles 2023-02-08 08:55:17,915 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/0188e80890ce408c90da25548b3d2a57 for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,915 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/7c18a1b1124b437aa6ac4ca4574bd6aa for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,915 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/284ce0dd25ac49459e03d8d9b0666ef9 for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/5477f14eaa574e989fe111de5de2a17f for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/573bbd30fdb14700861a01a224793061 for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/acb7ff0c34d84d7695b97d48daefdb93 for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/d63b5936006140559ed94f3e9779142c for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/dea8e16f3dd74291809c3038eb6c855c for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/3a17e09e335446e1b6ed1f591fb25d82 for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/7df916e5f1c94fa0a8259f67ef2aec54 for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/55af94394e4d4776a9a83012121fb840 for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/dc99f56e914e4cddbaa43f5992b35a20 for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/8eee8a31f2ca450389d1a3ee3f1350b0, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/dc241889a15f41ffbbbba991bb167763, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/eab5fe674d46495898c58cd9b8ca04bc, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/12f2df8055fc4e4da85427eb05d0bbff] hfiles 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/6980bf1172c44a8c8fb61eef66ebcc3e, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/60be5913ca21413f86163127d6eed588, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/a50110e6550e41a68e419e663403be7b, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/d7411eeea5ed4ed2936ab18ceb564590] hfiles 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/6980bf1172c44a8c8fb61eef66ebcc3e for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/8eee8a31f2ca450389d1a3ee3f1350b0 for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/729ebf7c577943268499efb8cb0a07ba, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/e5a1c334760a448d8a1d5fa71666c54c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/71e29344e20f4cd38dd724239183b532, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/299eac1ba6e746f28065bcf304ba36fd] hfiles 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/dc241889a15f41ffbbbba991bb167763 for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/60be5913ca21413f86163127d6eed588 for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/eab5fe674d46495898c58cd9b8ca04bc for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/729ebf7c577943268499efb8cb0a07ba for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/12f2df8055fc4e4da85427eb05d0bbff for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/a50110e6550e41a68e419e663403be7b for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/e5a1c334760a448d8a1d5fa71666c54c for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/71e29344e20f4cd38dd724239183b532 for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/d7411eeea5ed4ed2936ab18ceb564590 for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,921 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/299eac1ba6e746f28065bcf304ba36fd for snapshot=snaptb-1675846517299 2023-02-08 08:55:17,930 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. completed. 2023-02-08 08:55:17,930 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:17,930 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-02-08 08:55:17,944 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 08:55:17,944 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:17,944 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:17,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675846517299 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 08:55:17,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675846517299 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-08 08:55:17,945 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-02-08 08:55:18,328 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. completed. 2023-02-08 08:55:18,328 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:18,328 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-02-08 08:55:18,328 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. completed. 2023-02-08 08:55:18,328 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:18,328 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-02-08 08:55:18,328 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-02-08 08:55:18,328 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:18,328 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1675846517299' locally completed 2023-02-08 08:55:18,328 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1675846517299' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:55:18,330 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1675846517299' has notified controller of completion 2023-02-08 08:55:18,330 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675846517299/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:18,330 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:18,330 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1675846517299' completed. 2023-02-08 08:55:18,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1675846517299/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:18,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:18,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-08 08:55:18,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:18,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:18,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675846517299 2023-02-08 08:55:18,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:18,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:18,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675846517299 2023-02-08 08:55:18,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:18,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1675846517299' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:55:18,332 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'snaptb-1675846517299', counting down latch. Waiting for 0 more 2023-02-08 08:55:18,332 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1675846517299' execution completed 2023-02-08 08:55:18,332 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:18,332 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:18,332 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1675846517299 2023-02-08 08:55:18,332 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1675846517299including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-02-08 08:55:18,334 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1675846517299 2023-02-08 08:55:18,334 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1675846517299 2023-02-08 08:55:18,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1675846517299 2023-02-08 08:55:18,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:18,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-08 08:55:18,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1675846517299 2023-02-08 08:55:18,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1675846517299 2023-02-08 08:55:18,334 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-02-08 08:55:18,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:18,334 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1675846517299/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:18,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-02-08 08:55:18,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-08 08:55:18,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675846517299 2023-02-08 08:55:18,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1675846517299 2023-02-08 08:55:18,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:18,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675846517299 2023-02-08 08:55:18,335 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1675846517299/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:18,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:18,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:18,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1675846517299 2023-02-08 08:55:18,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:18,337 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1675846517299/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:18,337 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-02-08 08:55:18,337 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1675846517299 2023-02-08 08:55:18,337 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-02-08 08:55:18,337 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1675846517299 2023-02-08 08:55:18,337 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1675846517299 2023-02-08 08:55:18,337 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:18,337 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-02-08 08:55:18,337 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-02-08 08:55:18,337 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675846517299/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:18,337 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675846517299 2023-02-08 08:55:18,337 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1675846517299 2023-02-08 08:55:18,337 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/snaptb-1675846517299 because node does not exist (not an error) 2023-02-08 08:55:18,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-08 08:55:18,337 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-02-08 08:55:18,337 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1675846517299 2023-02-08 08:55:18,338 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1675846517299 2023-02-08 08:55:18,338 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-02-08 08:55:18,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-08 08:55:18,339 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020836ad795446ba45d68401de818d5a02f4, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208e0a819e1913c40aebbd0261f70920513, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208e31e783a362b4ccdb2c77b63c75f96cc, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208ee9b7e1c09a04572bfad5423c41f1272, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020842137fd8be5a4f54a55e79a58a213e9a, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302085696d16ea6fb44fe8532f61a0ed98c5c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230208845abc6337634d249733ad7bc14255fd, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302088573e6ea6b2449f2b300daa6fa8fb08d, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302082721e52113d24baca886c5b893723eee, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208547dd82594f94b97b01fec7e73f90b7e, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208df405c3a445f4c1cad9d8184abd4dd96, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208f5ce04e42d824d54a5cf228d419280d6] mob files 2023-02-08 08:55:18,339 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020836ad795446ba45d68401de818d5a02f4 2023-02-08 08:55:18,339 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208e0a819e1913c40aebbd0261f70920513 2023-02-08 08:55:18,339 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208e31e783a362b4ccdb2c77b63c75f96cc 2023-02-08 08:55:18,339 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208ee9b7e1c09a04572bfad5423c41f1272 2023-02-08 08:55:18,339 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020842137fd8be5a4f54a55e79a58a213e9a 2023-02-08 08:55:18,339 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302085696d16ea6fb44fe8532f61a0ed98c5c 2023-02-08 08:55:18,339 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230208845abc6337634d249733ad7bc14255fd 2023-02-08 08:55:18,339 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302088573e6ea6b2449f2b300daa6fa8fb08d 2023-02-08 08:55:18,339 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302082721e52113d24baca886c5b893723eee 2023-02-08 08:55:18,339 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208547dd82594f94b97b01fec7e73f90b7e 2023-02-08 08:55:18,339 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208df405c3a445f4c1cad9d8184abd4dd96 2023-02-08 08:55:18,339 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208f5ce04e42d824d54a5cf228d419280d6 2023-02-08 08:55:18,340 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208109b8e65fbf749c88b76ee104ef9f46e, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020820354c0d3bc849f09a31eec434c6b35d, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302086a575d9587e24ca2aadc4147fd222235, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208a67f884cad3d40238bc523a59405b027, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302082c30bcfc25054141a3239a271f481236, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020847397442cb504fcf889973f84a22c5f9, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302086f950b436eda41f0962a5eccf190e571, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302088fe72b3b4a7f48a3a8a02594a7eba7e2, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302080420b0823ab2472fbd59ab7c177a0363, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020854c51eb8b80e424697b30e0801b27a40, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020870729d0a550e41bb8659552a71588f5f, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208909c1d78aa524f7bb29ed96ee3bd693c] mob files 2023-02-08 08:55:18,340 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208109b8e65fbf749c88b76ee104ef9f46e 2023-02-08 08:55:18,340 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020820354c0d3bc849f09a31eec434c6b35d 2023-02-08 08:55:18,340 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302086a575d9587e24ca2aadc4147fd222235 2023-02-08 08:55:18,340 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208a67f884cad3d40238bc523a59405b027 2023-02-08 08:55:18,340 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302082c30bcfc25054141a3239a271f481236 2023-02-08 08:55:18,340 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020847397442cb504fcf889973f84a22c5f9 2023-02-08 08:55:18,340 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302086f950b436eda41f0962a5eccf190e571 2023-02-08 08:55:18,340 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302088fe72b3b4a7f48a3a8a02594a7eba7e2 2023-02-08 08:55:18,340 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302080420b0823ab2472fbd59ab7c177a0363 2023-02-08 08:55:18,340 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020854c51eb8b80e424697b30e0801b27a40 2023-02-08 08:55:18,340 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020870729d0a550e41bb8659552a71588f5f 2023-02-08 08:55:18,340 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208909c1d78aa524f7bb29ed96ee3bd693c 2023-02-08 08:55:18,445 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 08:55:18,446 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:18,446 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:18,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675846517299 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 08:55:18,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675846517299 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-08 08:55:18,446 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-02-08 08:55:18,745 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1675846517299 2023-02-08 08:55:18,747 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.hbase-snapshot/.tmp/snaptb-1675846517299 2023-02-08 08:55:19,189 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.hbase-snapshot/.tmp/snaptb-1675846517299 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.hbase-snapshot/snaptb-1675846517299 2023-02-08 08:55:19,447 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 08:55:19,447 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:19,447 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:19,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675846517299 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 08:55:19,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1675846517299 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-08 08:55:19,448 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2428): (#6) Sleeping: 2000ms while waiting for snapshot completion. 2023-02-08 08:55:20,004 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1675846517299 of table testCompactionWithHFileLink completed 2023-02-08 08:55:20,004 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.hbase-snapshot/.tmp/snaptb-1675846517299 2023-02-08 08:55:20,004 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.hbase-snapshot/.tmp/snaptb-1675846517299 2023-02-08 08:55:20,004 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1675846517299 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1675846517334Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1675846518745 (+1411 ms)Consolidate snapshot: snaptb-1675846517299 at 1675846518745Loading Region manifests for snaptb-1675846517299 at 1675846518746 (+1 ms)Writing data manifest for snaptb-1675846517299 at 1675846518755 (+9 ms)Verifying snapshot: snaptb-1675846517299 at 1675846519161 (+406 ms)Snapshot snaptb-1675846517299 of table testCompactionWithHFileLink completed at 1675846520004 (+843 ms) 2023-02-08 08:55:20,005 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-08 08:55:20,007 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 2.6710 sec 2023-02-08 08:55:21,448 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 08:55:21,448 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:21,448 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:21,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1675846517299 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 08:55:21,449 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1675846517299 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-02-08 08:55:21,476 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:21,476 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:21,476 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:21,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:21,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-08 08:55:21,478 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-08 08:55:21,478 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:21,478 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-08 08:55:21,478 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-08 08:55:21,479 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:21,479 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:21,480 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:21,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:21,480 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:21,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:21,480 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:21,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:21,480 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:21,481 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:21,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:21,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:21,481 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-08 08:55:21,481 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-08 08:55:21,481 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-08 08:55:21,481 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:21,481 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-08 08:55:21,482 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:21,482 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:21,482 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. started... 2023-02-08 08:55:21,482 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:21,482 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. started... 2023-02-08 08:55:21,482 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 08:55:21,482 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 665fcb26db73b9d2ce507202c21c1ade 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-08 08:55:21,482 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 79e77e74725bcdadb1e041ea443ff410 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-08 08:55:21,482 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:21,482 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. started... 2023-02-08 08:55:21,482 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 5f0bc0fd5600fb2bfd54b18349c52a4a 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-08 08:55:22,300 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family1/8f0f5e057f064961aa11a3144d4a2ebf 2023-02-08 08:55:22,300 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family1/b93be993c37849db9c895e2723131d92 2023-02-08 08:55:22,300 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family1/53d204cacda54f58802500bb5cb12bfc 2023-02-08 08:55:22,304 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8f0f5e057f064961aa11a3144d4a2ebf 2023-02-08 08:55:22,304 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 53d204cacda54f58802500bb5cb12bfc 2023-02-08 08:55:22,304 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b93be993c37849db9c895e2723131d92 2023-02-08 08:55:22,731 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family2/255a9f3c28d749cb8389b0cd6361a9ee 2023-02-08 08:55:22,734 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family2/0c9e25a9f0ba471d95ce2a13f22ef2a1 2023-02-08 08:55:22,734 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family2/b54b51f65ef84495b639d704a84fc7d1 2023-02-08 08:55:22,735 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 255a9f3c28d749cb8389b0cd6361a9ee 2023-02-08 08:55:22,735 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family1/53d204cacda54f58802500bb5cb12bfc as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/53d204cacda54f58802500bb5cb12bfc 2023-02-08 08:55:22,740 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0c9e25a9f0ba471d95ce2a13f22ef2a1 2023-02-08 08:55:22,740 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b54b51f65ef84495b639d704a84fc7d1 2023-02-08 08:55:22,741 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family1/b93be993c37849db9c895e2723131d92 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/b93be993c37849db9c895e2723131d92 2023-02-08 08:55:22,741 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family1/8f0f5e057f064961aa11a3144d4a2ebf as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/8f0f5e057f064961aa11a3144d4a2ebf 2023-02-08 08:55:22,742 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 53d204cacda54f58802500bb5cb12bfc 2023-02-08 08:55:22,742 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/53d204cacda54f58802500bb5cb12bfc, entries=3, sequenceid=24, filesize=5.0 K 2023-02-08 08:55:22,743 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family2/255a9f3c28d749cb8389b0cd6361a9ee as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/255a9f3c28d749cb8389b0cd6361a9ee 2023-02-08 08:55:22,745 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8f0f5e057f064961aa11a3144d4a2ebf 2023-02-08 08:55:22,745 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/8f0f5e057f064961aa11a3144d4a2ebf, entries=3, sequenceid=24, filesize=5.0 K 2023-02-08 08:55:22,745 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b93be993c37849db9c895e2723131d92 2023-02-08 08:55:22,745 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/b93be993c37849db9c895e2723131d92, entries=3, sequenceid=24, filesize=5.0 K 2023-02-08 08:55:22,745 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family2/b54b51f65ef84495b639d704a84fc7d1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/b54b51f65ef84495b639d704a84fc7d1 2023-02-08 08:55:22,746 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family2/0c9e25a9f0ba471d95ce2a13f22ef2a1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/0c9e25a9f0ba471d95ce2a13f22ef2a1 2023-02-08 08:55:22,746 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 255a9f3c28d749cb8389b0cd6361a9ee 2023-02-08 08:55:22,746 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/255a9f3c28d749cb8389b0cd6361a9ee, entries=1, sequenceid=24, filesize=5.0 K 2023-02-08 08:55:22,747 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 5f0bc0fd5600fb2bfd54b18349c52a4a in 1265ms, sequenceid=24, compaction requested=true 2023-02-08 08:55:22,747 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 5f0bc0fd5600fb2bfd54b18349c52a4a: 2023-02-08 08:55:22,747 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:22,747 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 08:55:22,749 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b54b51f65ef84495b639d704a84fc7d1 2023-02-08 08:55:22,749 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/b54b51f65ef84495b639d704a84fc7d1, entries=1, sequenceid=24, filesize=5.0 K 2023-02-08 08:55:22,749 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0c9e25a9f0ba471d95ce2a13f22ef2a1 2023-02-08 08:55:22,749 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/0c9e25a9f0ba471d95ce2a13f22ef2a1, entries=1, sequenceid=24, filesize=5.0 K 2023-02-08 08:55:22,749 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 79e77e74725bcdadb1e041ea443ff410 in 1267ms, sequenceid=24, compaction requested=true 2023-02-08 08:55:22,750 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 79e77e74725bcdadb1e041ea443ff410: 2023-02-08 08:55:22,750 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:22,750 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 08:55:22,750 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 665fcb26db73b9d2ce507202c21c1ade in 1268ms, sequenceid=24, compaction requested=true 2023-02-08 08:55:22,750 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 665fcb26db73b9d2ce507202c21c1ade: 2023-02-08 08:55:22,750 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:22,750 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 08:55:22,750 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 08:55:22,750 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:22,750 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-08 08:55:22,750 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-08 08:55:22,752 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:22,752 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:22,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:22,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:22,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:22,752 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:22,752 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:22,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:22,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:22,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:22,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:22,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:22,753 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-08 08:55:22,753 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-08 08:55:22,753 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@626baf5e[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:22,753 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:22,762 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:22,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:22,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:22,763 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-08 08:55:22,763 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-08 08:55:22,763 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:55:22,763 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:22,763 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:22,764 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-08 08:55:22,764 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:22,764 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:22,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:22,764 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-08 08:55:22,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:22,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:22,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:22,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:22,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:22,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:22,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:22,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:22,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:22,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:55:22,766 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-08 08:55:22,766 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-08 08:55:22,766 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:22,766 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:22,766 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-08 08:55:22,766 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:22,767 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:22,767 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:22,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:22,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:22,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:22,767 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:22,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:22,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:22,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:22,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:22,768 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:22,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:22,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:22,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:22,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:22,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:22,768 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:22,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:22,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:22,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-08 08:55:22,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:22,770 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:22,770 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:22,770 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:22,770 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:22,770 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-08 08:55:22,771 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:22,770 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-08 08:55:22,770 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:22,771 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:22,771 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:22,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:22,771 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-08 08:55:22,771 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-08 08:55:22,771 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-08 08:55:22,771 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:22,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:22,771 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-08 08:55:22,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:22,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:22,871 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:22,871 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:22,871 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:22,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:22,873 DEBUG [Listener at localhost/40763] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 08:55:22,873 DEBUG [Listener at localhost/40763] regionserver.HStore(1912): 665fcb26db73b9d2ce507202c21c1ade/family1 is initiating major compaction (all files) 2023-02-08 08:55:22,873 INFO [Listener at localhost/40763] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:55:22,873 INFO [Listener at localhost/40763] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:55:22,873 INFO [Listener at localhost/40763] regionserver.HRegion(2258): Starting compaction of 665fcb26db73b9d2ce507202c21c1ade/family1 in testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:22,873 INFO [Listener at localhost/40763] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/7c18a1b1124b437aa6ac4ca4574bd6aa, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/573bbd30fdb14700861a01a224793061, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/d63b5936006140559ed94f3e9779142c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/3a17e09e335446e1b6ed1f591fb25d82, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/b93be993c37849db9c895e2723131d92] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp, totalSize=26.7 K 2023-02-08 08:55:22,874 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 7c18a1b1124b437aa6ac4ca4574bd6aa, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846510550 2023-02-08 08:55:22,874 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 573bbd30fdb14700861a01a224793061, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846511936 2023-02-08 08:55:22,875 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting d63b5936006140559ed94f3e9779142c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846513711 2023-02-08 08:55:22,875 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 3a17e09e335446e1b6ed1f591fb25d82, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846515502 2023-02-08 08:55:22,875 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting b93be993c37849db9c895e2723131d92, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-08 08:55:22,887 INFO [Listener at localhost/40763] throttle.PressureAwareThroughputController(145): 665fcb26db73b9d2ce507202c21c1ade#family1#compaction#107 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:23,294 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230208019610ae7936458a98ce9254ab0c3f84_del 2023-02-08 08:55:23,295 INFO [Listener at localhost/40763] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208019610ae7936458a98ce9254ab0c3f84_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230208019610ae7936458a98ce9254ab0c3f84_del 2023-02-08 08:55:24,107 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for efae5049a5ce45f0b4753e1951139145 2023-02-08 08:55:24,108 DEBUG [Listener at localhost/40763] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family1/efae5049a5ce45f0b4753e1951139145 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/efae5049a5ce45f0b4753e1951139145 2023-02-08 08:55:24,111 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for efae5049a5ce45f0b4753e1951139145 2023-02-08 08:55:24,112 INFO [Listener at localhost/40763] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 665fcb26db73b9d2ce507202c21c1ade/family1 of 665fcb26db73b9d2ce507202c21c1ade into efae5049a5ce45f0b4753e1951139145(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 08:55:24,112 DEBUG [Listener at localhost/40763] regionserver.HRegion(2288): Compaction status journal for 665fcb26db73b9d2ce507202c21c1ade: 2023-02-08 08:55:24,112 DEBUG [Listener at localhost/40763] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 08:55:24,112 DEBUG [Listener at localhost/40763] regionserver.HStore(1912): 665fcb26db73b9d2ce507202c21c1ade/family2 is initiating major compaction (all files) 2023-02-08 08:55:24,112 INFO [Listener at localhost/40763] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:55:24,112 INFO [Listener at localhost/40763] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:55:24,113 INFO [Listener at localhost/40763] regionserver.HRegion(2258): Starting compaction of 665fcb26db73b9d2ce507202c21c1ade/family2 in testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:24,113 INFO [Listener at localhost/40763] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/729ebf7c577943268499efb8cb0a07ba, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/e5a1c334760a448d8a1d5fa71666c54c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/71e29344e20f4cd38dd724239183b532, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/299eac1ba6e746f28065bcf304ba36fd, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/0c9e25a9f0ba471d95ce2a13f22ef2a1] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp, totalSize=25.4 K 2023-02-08 08:55:24,114 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 729ebf7c577943268499efb8cb0a07ba, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846510550 2023-02-08 08:55:24,114 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/7c18a1b1124b437aa6ac4ca4574bd6aa, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/573bbd30fdb14700861a01a224793061, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/d63b5936006140559ed94f3e9779142c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/3a17e09e335446e1b6ed1f591fb25d82, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/b93be993c37849db9c895e2723131d92] to archive 2023-02-08 08:55:24,114 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting e5a1c334760a448d8a1d5fa71666c54c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846511936 2023-02-08 08:55:24,114 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 71e29344e20f4cd38dd724239183b532, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846513711 2023-02-08 08:55:24,114 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:24,115 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 299eac1ba6e746f28065bcf304ba36fd, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846515502 2023-02-08 08:55:24,115 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 0c9e25a9f0ba471d95ce2a13f22ef2a1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-08 08:55:24,116 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/7c18a1b1124b437aa6ac4ca4574bd6aa to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/7c18a1b1124b437aa6ac4ca4574bd6aa 2023-02-08 08:55:24,117 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/573bbd30fdb14700861a01a224793061 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/573bbd30fdb14700861a01a224793061 2023-02-08 08:55:24,118 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/d63b5936006140559ed94f3e9779142c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/d63b5936006140559ed94f3e9779142c 2023-02-08 08:55:24,119 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/3a17e09e335446e1b6ed1f591fb25d82 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/3a17e09e335446e1b6ed1f591fb25d82 2023-02-08 08:55:24,120 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/b93be993c37849db9c895e2723131d92 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/b93be993c37849db9c895e2723131d92 2023-02-08 08:55:24,127 INFO [Listener at localhost/40763] throttle.PressureAwareThroughputController(145): 665fcb26db73b9d2ce507202c21c1ade#family2#compaction#108 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:24,535 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302087b26c8b1b47740e8b7fffa6d100ab412_del 2023-02-08 08:55:24,535 INFO [Listener at localhost/40763] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302087b26c8b1b47740e8b7fffa6d100ab412_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302087b26c8b1b47740e8b7fffa6d100ab412_del 2023-02-08 08:55:25,347 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b24789fb11c54561aecf5266a1dab6c9 2023-02-08 08:55:25,348 DEBUG [Listener at localhost/40763] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family2/b24789fb11c54561aecf5266a1dab6c9 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/b24789fb11c54561aecf5266a1dab6c9 2023-02-08 08:55:25,351 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b24789fb11c54561aecf5266a1dab6c9 2023-02-08 08:55:25,352 INFO [Listener at localhost/40763] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 665fcb26db73b9d2ce507202c21c1ade/family2 of 665fcb26db73b9d2ce507202c21c1ade into b24789fb11c54561aecf5266a1dab6c9(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 08:55:25,352 DEBUG [Listener at localhost/40763] regionserver.HRegion(2288): Compaction status journal for 665fcb26db73b9d2ce507202c21c1ade: 2023-02-08 08:55:25,352 DEBUG [Listener at localhost/40763] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 08:55:25,352 DEBUG [Listener at localhost/40763] regionserver.HStore(1912): 5f0bc0fd5600fb2bfd54b18349c52a4a/family1 is initiating major compaction (all files) 2023-02-08 08:55:25,352 INFO [Listener at localhost/40763] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:55:25,352 INFO [Listener at localhost/40763] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:55:25,352 INFO [Listener at localhost/40763] regionserver.HRegion(2258): Starting compaction of 5f0bc0fd5600fb2bfd54b18349c52a4a/family1 in testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:25,352 INFO [Listener at localhost/40763] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/284ce0dd25ac49459e03d8d9b0666ef9, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/5477f14eaa574e989fe111de5de2a17f, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/dea8e16f3dd74291809c3038eb6c855c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/55af94394e4d4776a9a83012121fb840, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/53d204cacda54f58802500bb5cb12bfc] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp, totalSize=26.7 K 2023-02-08 08:55:25,353 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 284ce0dd25ac49459e03d8d9b0666ef9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846510549 2023-02-08 08:55:25,353 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 5477f14eaa574e989fe111de5de2a17f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846511936 2023-02-08 08:55:25,353 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting dea8e16f3dd74291809c3038eb6c855c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846513711 2023-02-08 08:55:25,353 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 55af94394e4d4776a9a83012121fb840, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846515501 2023-02-08 08:55:25,354 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 53d204cacda54f58802500bb5cb12bfc, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-08 08:55:25,363 INFO [Listener at localhost/40763] throttle.PressureAwareThroughputController(145): 5f0bc0fd5600fb2bfd54b18349c52a4a#family1#compaction#109 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:25,413 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/729ebf7c577943268499efb8cb0a07ba, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/e5a1c334760a448d8a1d5fa71666c54c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/71e29344e20f4cd38dd724239183b532, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/299eac1ba6e746f28065bcf304ba36fd, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/0c9e25a9f0ba471d95ce2a13f22ef2a1] to archive 2023-02-08 08:55:25,413 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:25,415 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/729ebf7c577943268499efb8cb0a07ba to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/729ebf7c577943268499efb8cb0a07ba 2023-02-08 08:55:25,416 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/e5a1c334760a448d8a1d5fa71666c54c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/e5a1c334760a448d8a1d5fa71666c54c 2023-02-08 08:55:25,417 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/71e29344e20f4cd38dd724239183b532 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/71e29344e20f4cd38dd724239183b532 2023-02-08 08:55:25,417 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/299eac1ba6e746f28065bcf304ba36fd to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/299eac1ba6e746f28065bcf304ba36fd 2023-02-08 08:55:25,418 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/0c9e25a9f0ba471d95ce2a13f22ef2a1 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/0c9e25a9f0ba471d95ce2a13f22ef2a1 2023-02-08 08:55:25,772 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023020897302e68d68d47de8bf71bae9199364b_del 2023-02-08 08:55:25,773 INFO [Listener at localhost/40763] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020897302e68d68d47de8bf71bae9199364b_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020897302e68d68d47de8bf71bae9199364b_del 2023-02-08 08:55:26,586 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cf8dba0dd3b944b8b254e36bb74fe986 2023-02-08 08:55:26,586 DEBUG [Listener at localhost/40763] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family1/cf8dba0dd3b944b8b254e36bb74fe986 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/cf8dba0dd3b944b8b254e36bb74fe986 2023-02-08 08:55:26,590 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cf8dba0dd3b944b8b254e36bb74fe986 2023-02-08 08:55:26,591 INFO [Listener at localhost/40763] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 5f0bc0fd5600fb2bfd54b18349c52a4a/family1 of 5f0bc0fd5600fb2bfd54b18349c52a4a into cf8dba0dd3b944b8b254e36bb74fe986(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 08:55:26,591 DEBUG [Listener at localhost/40763] regionserver.HRegion(2288): Compaction status journal for 5f0bc0fd5600fb2bfd54b18349c52a4a: 2023-02-08 08:55:26,591 DEBUG [Listener at localhost/40763] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 08:55:26,591 DEBUG [Listener at localhost/40763] regionserver.HStore(1912): 5f0bc0fd5600fb2bfd54b18349c52a4a/family2 is initiating major compaction (all files) 2023-02-08 08:55:26,591 INFO [Listener at localhost/40763] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:55:26,591 INFO [Listener at localhost/40763] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:55:26,591 INFO [Listener at localhost/40763] regionserver.HRegion(2258): Starting compaction of 5f0bc0fd5600fb2bfd54b18349c52a4a/family2 in testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:26,591 INFO [Listener at localhost/40763] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/8eee8a31f2ca450389d1a3ee3f1350b0, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/dc241889a15f41ffbbbba991bb167763, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/eab5fe674d46495898c58cd9b8ca04bc, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/12f2df8055fc4e4da85427eb05d0bbff, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/255a9f3c28d749cb8389b0cd6361a9ee] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp, totalSize=25.4 K 2023-02-08 08:55:26,592 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 8eee8a31f2ca450389d1a3ee3f1350b0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846510549 2023-02-08 08:55:26,592 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting dc241889a15f41ffbbbba991bb167763, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846511936 2023-02-08 08:55:26,593 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting eab5fe674d46495898c58cd9b8ca04bc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846513711 2023-02-08 08:55:26,593 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 12f2df8055fc4e4da85427eb05d0bbff, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846515501 2023-02-08 08:55:26,593 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 255a9f3c28d749cb8389b0cd6361a9ee, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-08 08:55:26,604 INFO [Listener at localhost/40763] throttle.PressureAwareThroughputController(145): 5f0bc0fd5600fb2bfd54b18349c52a4a#family2#compaction#110 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:26,614 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/284ce0dd25ac49459e03d8d9b0666ef9, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/5477f14eaa574e989fe111de5de2a17f, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/dea8e16f3dd74291809c3038eb6c855c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/55af94394e4d4776a9a83012121fb840, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/53d204cacda54f58802500bb5cb12bfc] to archive 2023-02-08 08:55:26,614 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:26,616 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/284ce0dd25ac49459e03d8d9b0666ef9 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/284ce0dd25ac49459e03d8d9b0666ef9 2023-02-08 08:55:26,616 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/5477f14eaa574e989fe111de5de2a17f to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/5477f14eaa574e989fe111de5de2a17f 2023-02-08 08:55:26,617 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/dea8e16f3dd74291809c3038eb6c855c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/dea8e16f3dd74291809c3038eb6c855c 2023-02-08 08:55:26,618 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/55af94394e4d4776a9a83012121fb840 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/55af94394e4d4776a9a83012121fb840 2023-02-08 08:55:26,619 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/53d204cacda54f58802500bb5cb12bfc to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/53d204cacda54f58802500bb5cb12bfc 2023-02-08 08:55:27,012 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230208887265ec3b934b5fb6965e891b03f88f_del 2023-02-08 08:55:27,012 INFO [Listener at localhost/40763] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208887265ec3b934b5fb6965e891b03f88f_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208887265ec3b934b5fb6965e891b03f88f_del 2023-02-08 08:55:27,825 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e1af45530f26453281c5295607de8334 2023-02-08 08:55:27,825 DEBUG [Listener at localhost/40763] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family2/e1af45530f26453281c5295607de8334 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/e1af45530f26453281c5295607de8334 2023-02-08 08:55:27,829 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e1af45530f26453281c5295607de8334 2023-02-08 08:55:27,830 INFO [Listener at localhost/40763] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 5f0bc0fd5600fb2bfd54b18349c52a4a/family2 of 5f0bc0fd5600fb2bfd54b18349c52a4a into e1af45530f26453281c5295607de8334(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 08:55:27,830 DEBUG [Listener at localhost/40763] regionserver.HRegion(2288): Compaction status journal for 5f0bc0fd5600fb2bfd54b18349c52a4a: 2023-02-08 08:55:27,830 DEBUG [Listener at localhost/40763] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 08:55:27,830 DEBUG [Listener at localhost/40763] regionserver.HStore(1912): 79e77e74725bcdadb1e041ea443ff410/family1 is initiating major compaction (all files) 2023-02-08 08:55:27,830 INFO [Listener at localhost/40763] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:55:27,830 INFO [Listener at localhost/40763] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:55:27,830 INFO [Listener at localhost/40763] regionserver.HRegion(2258): Starting compaction of 79e77e74725bcdadb1e041ea443ff410/family1 in testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:27,830 INFO [Listener at localhost/40763] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/0188e80890ce408c90da25548b3d2a57, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/acb7ff0c34d84d7695b97d48daefdb93, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/7df916e5f1c94fa0a8259f67ef2aec54, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/dc99f56e914e4cddbaa43f5992b35a20, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/8f0f5e057f064961aa11a3144d4a2ebf] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp, totalSize=26.7 K 2023-02-08 08:55:27,831 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 0188e80890ce408c90da25548b3d2a57, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846510548 2023-02-08 08:55:27,831 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting acb7ff0c34d84d7695b97d48daefdb93, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846511935 2023-02-08 08:55:27,831 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 7df916e5f1c94fa0a8259f67ef2aec54, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846513710 2023-02-08 08:55:27,831 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting dc99f56e914e4cddbaa43f5992b35a20, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846515501 2023-02-08 08:55:27,832 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 8f0f5e057f064961aa11a3144d4a2ebf, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-08 08:55:27,840 INFO [Listener at localhost/40763] throttle.PressureAwareThroughputController(145): 79e77e74725bcdadb1e041ea443ff410#family1#compaction#111 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:27,913 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/8eee8a31f2ca450389d1a3ee3f1350b0, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/dc241889a15f41ffbbbba991bb167763, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/eab5fe674d46495898c58cd9b8ca04bc, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/12f2df8055fc4e4da85427eb05d0bbff, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/255a9f3c28d749cb8389b0cd6361a9ee] to archive 2023-02-08 08:55:27,913 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:27,915 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/8eee8a31f2ca450389d1a3ee3f1350b0 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/8eee8a31f2ca450389d1a3ee3f1350b0 2023-02-08 08:55:27,916 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/dc241889a15f41ffbbbba991bb167763 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/dc241889a15f41ffbbbba991bb167763 2023-02-08 08:55:27,917 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/eab5fe674d46495898c58cd9b8ca04bc to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/eab5fe674d46495898c58cd9b8ca04bc 2023-02-08 08:55:27,917 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/12f2df8055fc4e4da85427eb05d0bbff to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/12f2df8055fc4e4da85427eb05d0bbff 2023-02-08 08:55:27,918 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/255a9f3c28d749cb8389b0cd6361a9ee to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/255a9f3c28d749cb8389b0cd6361a9ee 2023-02-08 08:55:28,248 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302080d8598f4b4cc48f1b6e3ccb5de6f7706_del 2023-02-08 08:55:28,248 INFO [Listener at localhost/40763] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302080d8598f4b4cc48f1b6e3ccb5de6f7706_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302080d8598f4b4cc48f1b6e3ccb5de6f7706_del 2023-02-08 08:55:29,061 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 106075da0fe448b8905997054aa934fc 2023-02-08 08:55:29,062 DEBUG [Listener at localhost/40763] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family1/106075da0fe448b8905997054aa934fc as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/106075da0fe448b8905997054aa934fc 2023-02-08 08:55:29,065 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 106075da0fe448b8905997054aa934fc 2023-02-08 08:55:29,066 INFO [Listener at localhost/40763] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 79e77e74725bcdadb1e041ea443ff410/family1 of 79e77e74725bcdadb1e041ea443ff410 into 106075da0fe448b8905997054aa934fc(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 08:55:29,066 DEBUG [Listener at localhost/40763] regionserver.HRegion(2288): Compaction status journal for 79e77e74725bcdadb1e041ea443ff410: 2023-02-08 08:55:29,066 DEBUG [Listener at localhost/40763] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 08:55:29,067 DEBUG [Listener at localhost/40763] regionserver.HStore(1912): 79e77e74725bcdadb1e041ea443ff410/family2 is initiating major compaction (all files) 2023-02-08 08:55:29,067 INFO [Listener at localhost/40763] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:55:29,067 INFO [Listener at localhost/40763] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:55:29,067 INFO [Listener at localhost/40763] regionserver.HRegion(2258): Starting compaction of 79e77e74725bcdadb1e041ea443ff410/family2 in testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:29,067 INFO [Listener at localhost/40763] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/6980bf1172c44a8c8fb61eef66ebcc3e, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/60be5913ca21413f86163127d6eed588, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/a50110e6550e41a68e419e663403be7b, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/d7411eeea5ed4ed2936ab18ceb564590, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/b54b51f65ef84495b639d704a84fc7d1] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp, totalSize=25.4 K 2023-02-08 08:55:29,067 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 6980bf1172c44a8c8fb61eef66ebcc3e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846510548 2023-02-08 08:55:29,068 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 60be5913ca21413f86163127d6eed588, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846511935 2023-02-08 08:55:29,068 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting a50110e6550e41a68e419e663403be7b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846513710 2023-02-08 08:55:29,068 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting d7411eeea5ed4ed2936ab18ceb564590, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846515501 2023-02-08 08:55:29,069 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting b54b51f65ef84495b639d704a84fc7d1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-08 08:55:29,079 INFO [Listener at localhost/40763] throttle.PressureAwareThroughputController(145): 79e77e74725bcdadb1e041ea443ff410#family2#compaction#112 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:29,117 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/0188e80890ce408c90da25548b3d2a57, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/acb7ff0c34d84d7695b97d48daefdb93, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/7df916e5f1c94fa0a8259f67ef2aec54, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/dc99f56e914e4cddbaa43f5992b35a20, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/8f0f5e057f064961aa11a3144d4a2ebf] to archive 2023-02-08 08:55:29,118 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:29,119 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/0188e80890ce408c90da25548b3d2a57 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/0188e80890ce408c90da25548b3d2a57 2023-02-08 08:55:29,120 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/acb7ff0c34d84d7695b97d48daefdb93 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/acb7ff0c34d84d7695b97d48daefdb93 2023-02-08 08:55:29,121 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/7df916e5f1c94fa0a8259f67ef2aec54 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/7df916e5f1c94fa0a8259f67ef2aec54 2023-02-08 08:55:29,122 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/dc99f56e914e4cddbaa43f5992b35a20 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/dc99f56e914e4cddbaa43f5992b35a20 2023-02-08 08:55:29,122 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/8f0f5e057f064961aa11a3144d4a2ebf to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/8f0f5e057f064961aa11a3144d4a2ebf 2023-02-08 08:55:29,493 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230208eadaa52fb7544e3798a0b7ce5b5d7579_del 2023-02-08 08:55:29,493 INFO [Listener at localhost/40763] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208eadaa52fb7544e3798a0b7ce5b5d7579_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208eadaa52fb7544e3798a0b7ce5b5d7579_del 2023-02-08 08:55:30,306 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03fe9ec355c64fee916a277921a47304 2023-02-08 08:55:30,307 DEBUG [Listener at localhost/40763] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family2/03fe9ec355c64fee916a277921a47304 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/03fe9ec355c64fee916a277921a47304 2023-02-08 08:55:30,310 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 03fe9ec355c64fee916a277921a47304 2023-02-08 08:55:30,311 INFO [Listener at localhost/40763] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 79e77e74725bcdadb1e041ea443ff410/family2 of 79e77e74725bcdadb1e041ea443ff410 into 03fe9ec355c64fee916a277921a47304(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 08:55:30,311 DEBUG [Listener at localhost/40763] regionserver.HRegion(2288): Compaction status journal for 79e77e74725bcdadb1e041ea443ff410: 2023-02-08 08:55:30,316 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 08:55:30,319 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 08:55:30,321 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/6980bf1172c44a8c8fb61eef66ebcc3e, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/60be5913ca21413f86163127d6eed588, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/a50110e6550e41a68e419e663403be7b, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/d7411eeea5ed4ed2936ab18ceb564590, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/b54b51f65ef84495b639d704a84fc7d1] to archive 2023-02-08 08:55:30,322 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:30,323 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 08:55:30,323 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/6980bf1172c44a8c8fb61eef66ebcc3e to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/6980bf1172c44a8c8fb61eef66ebcc3e 2023-02-08 08:55:30,325 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 08:55:30,325 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/60be5913ca21413f86163127d6eed588 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/60be5913ca21413f86163127d6eed588 2023-02-08 08:55:30,327 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/a50110e6550e41a68e419e663403be7b to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/a50110e6550e41a68e419e663403be7b 2023-02-08 08:55:30,329 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/d7411eeea5ed4ed2936ab18ceb564590 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/d7411eeea5ed4ed2936ab18ceb564590 2023-02-08 08:55:30,331 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/b54b51f65ef84495b639d704a84fc7d1 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/b54b51f65ef84495b639d704a84fc7d1 2023-02-08 08:55:30,334 INFO [Listener at localhost/40763] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-08 08:55:30,371 INFO [Listener at localhost/40763] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-02-08 08:55:30,374 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302080d8598f4b4cc48f1b6e3ccb5de6f7706_del 2023-02-08 08:55:30,377 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023020897302e68d68d47de8bf71bae9199364b_del 2023-02-08 08:55:30,379 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230208019610ae7936458a98ce9254ab0c3f84_del 2023-02-08 08:55:30,379 INFO [Listener at localhost/40763] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-02-08 08:55:30,380 DEBUG [Listener at localhost/40763] zookeeper.ReadOnlyZKClient(139): Connect 0x5af1d7e5 to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:55:30,385 DEBUG [Listener at localhost/40763] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@460b5b99, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:55:30,386 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 08:55:30,386 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 08:55:30,386 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 08:55:30,460 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302089e6aa4c566024b4ebf039193c398e1cb to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302089e6aa4c566024b4ebf039193c398e1cb 2023-02-08 08:55:30,460 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208b7e204bb7ae44f07a6af6c099bd6c36f to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208b7e204bb7ae44f07a6af6c099bd6c36f 2023-02-08 08:55:30,461 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 08:55:30,461 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 08:55:30,462 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208fa2dcb1473ac4c99b6141964fea7b619 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230208fa2dcb1473ac4c99b6141964fea7b619 2023-02-08 08:55:30,464 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 08:55:30,475 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:55:30,481 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60778, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:55:30,486 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:30,486 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:30,486 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:30,486 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:30,487 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 08:55:30,489 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58026, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 08:55:30,489 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:30,489 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:30,490 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:30,490 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:30,503 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230208/family1/e46f1d9b735640459a6e88469f176c8b first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-02-08 08:55:30,503 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230208/family1/525dd81d9dc84f65ad6ff6eaeacaca5e first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-02-08 08:55:30,503 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230208/family1/3ef007f13bd94c74a361f46a672b9373 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-02-08 08:55:30,505 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230208/family1/e46f1d9b735640459a6e88469f176c8b}] 2023-02-08 08:55:30,505 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230208/family1/3ef007f13bd94c74a361f46a672b9373}] 2023-02-08 08:55:30,505 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230208/family1/525dd81d9dc84f65ad6ff6eaeacaca5e}] 2023-02-08 08:55:30,510 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230208/family1/e46f1d9b735640459a6e88469f176c8b for inclusion in 5f0bc0fd5600fb2bfd54b18349c52a4a/family1 2023-02-08 08:55:30,510 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230208/family1/3ef007f13bd94c74a361f46a672b9373 for inclusion in 665fcb26db73b9d2ce507202c21c1ade/family1 2023-02-08 08:55:30,510 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230208/family1/525dd81d9dc84f65ad6ff6eaeacaca5e for inclusion in 79e77e74725bcdadb1e041ea443ff410/family1 2023-02-08 08:55:30,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-02-08 08:55:30,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42239] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-08 08:55:30,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-02-08 08:55:30,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-02-08 08:55:30,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last=1 2023-02-08 08:55:30,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for 5f0bc0fd5600fb2bfd54b18349c52a4a: 2023-02-08 08:55:30,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-08 08:55:30,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for 79e77e74725bcdadb1e041ea443ff410: 2023-02-08 08:55:30,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for 665fcb26db73b9d2ce507202c21c1ade: 2023-02-08 08:55:30,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230208/family1/e46f1d9b735640459a6e88469f176c8b to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__j2op8p8m6lfbabj8hblcqqqb74hdgjk52ns023duimehdr9l14vqd73cdmedetnb/family1/e46f1d9b735640459a6e88469f176c8b 2023-02-08 08:55:30,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230208/family1/3ef007f13bd94c74a361f46a672b9373 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__h8js5ip0b6199itgnii8cqnb7itmlupb6g14sfieoi2annjdkj9dmkd88ksgvc5k/family1/3ef007f13bd94c74a361f46a672b9373 2023-02-08 08:55:30,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230208/family1/525dd81d9dc84f65ad6ff6eaeacaca5e to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__hisji6pqi4e2lqn8eln4d8jje3q9ruit2klk5g1j2hsb11aemaefvad1dtctffnd/family1/525dd81d9dc84f65ad6ff6eaeacaca5e 2023-02-08 08:55:30,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__j2op8p8m6lfbabj8hblcqqqb74hdgjk52ns023duimehdr9l14vqd73cdmedetnb/family1/e46f1d9b735640459a6e88469f176c8b as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/26ec5f06dba04baab65d99a31660e218_SeqId_29_ 2023-02-08 08:55:30,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__h8js5ip0b6199itgnii8cqnb7itmlupb6g14sfieoi2annjdkj9dmkd88ksgvc5k/family1/3ef007f13bd94c74a361f46a672b9373 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/f3e93bc435324838999f46ea05b945c9_SeqId_29_ 2023-02-08 08:55:30,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__hisji6pqi4e2lqn8eln4d8jje3q9ruit2klk5g1j2hsb11aemaefvad1dtctffnd/family1/525dd81d9dc84f65ad6ff6eaeacaca5e as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/13de81ef75754182911e7c924c1662c1_SeqId_29_ 2023-02-08 08:55:30,515 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__j2op8p8m6lfbabj8hblcqqqb74hdgjk52ns023duimehdr9l14vqd73cdmedetnb/family1/e46f1d9b735640459a6e88469f176c8b into 5f0bc0fd5600fb2bfd54b18349c52a4a/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/26ec5f06dba04baab65d99a31660e218_SeqId_29_ - updating store file list. 2023-02-08 08:55:30,516 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__h8js5ip0b6199itgnii8cqnb7itmlupb6g14sfieoi2annjdkj9dmkd88ksgvc5k/family1/3ef007f13bd94c74a361f46a672b9373 into 665fcb26db73b9d2ce507202c21c1ade/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/f3e93bc435324838999f46ea05b945c9_SeqId_29_ - updating store file list. 2023-02-08 08:55:30,516 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__hisji6pqi4e2lqn8eln4d8jje3q9ruit2klk5g1j2hsb11aemaefvad1dtctffnd/family1/525dd81d9dc84f65ad6ff6eaeacaca5e into 79e77e74725bcdadb1e041ea443ff410/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/13de81ef75754182911e7c924c1662c1_SeqId_29_ - updating store file list. 2023-02-08 08:55:30,518 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/26ec5f06dba04baab65d99a31660e218_SeqId_29_ into 5f0bc0fd5600fb2bfd54b18349c52a4a/family1 2023-02-08 08:55:30,518 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/13de81ef75754182911e7c924c1662c1_SeqId_29_ into 79e77e74725bcdadb1e041ea443ff410/family1 2023-02-08 08:55:30,519 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/f3e93bc435324838999f46ea05b945c9_SeqId_29_ into 665fcb26db73b9d2ce507202c21c1ade/family1 2023-02-08 08:55:30,519 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__h8js5ip0b6199itgnii8cqnb7itmlupb6g14sfieoi2annjdkj9dmkd88ksgvc5k/family1/3ef007f13bd94c74a361f46a672b9373 into 665fcb26db73b9d2ce507202c21c1ade/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/f3e93bc435324838999f46ea05b945c9_SeqId_29_) 2023-02-08 08:55:30,519 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__j2op8p8m6lfbabj8hblcqqqb74hdgjk52ns023duimehdr9l14vqd73cdmedetnb/family1/e46f1d9b735640459a6e88469f176c8b into 5f0bc0fd5600fb2bfd54b18349c52a4a/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/26ec5f06dba04baab65d99a31660e218_SeqId_29_) 2023-02-08 08:55:30,519 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__hisji6pqi4e2lqn8eln4d8jje3q9ruit2klk5g1j2hsb11aemaefvad1dtctffnd/family1/525dd81d9dc84f65ad6ff6eaeacaca5e into 79e77e74725bcdadb1e041ea443ff410/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/13de81ef75754182911e7c924c1662c1_SeqId_29_) 2023-02-08 08:55:30,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__h8js5ip0b6199itgnii8cqnb7itmlupb6g14sfieoi2annjdkj9dmkd88ksgvc5k/family1/3ef007f13bd94c74a361f46a672b9373 2023-02-08 08:55:30,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__hisji6pqi4e2lqn8eln4d8jje3q9ruit2klk5g1j2hsb11aemaefvad1dtctffnd/family1/525dd81d9dc84f65ad6ff6eaeacaca5e 2023-02-08 08:55:30,519 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__j2op8p8m6lfbabj8hblcqqqb74hdgjk52ns023duimehdr9l14vqd73cdmedetnb/family1/e46f1d9b735640459a6e88469f176c8b 2023-02-08 08:55:30,521 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 08:55:30,521 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 08:55:30,521 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 08:55:30,522 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:30,522 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:30,522 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:30,523 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020842137fd8be5a4f54a55e79a58a213e9a to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023020842137fd8be5a4f54a55e79a58a213e9a 2023-02-08 08:55:30,523 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020836ad795446ba45d68401de818d5a02f4 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020836ad795446ba45d68401de818d5a02f4 2023-02-08 08:55:30,523 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302082721e52113d24baca886c5b893723eee to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302082721e52113d24baca886c5b893723eee 2023-02-08 08:55:30,524 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302085696d16ea6fb44fe8532f61a0ed98c5c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302085696d16ea6fb44fe8532f61a0ed98c5c 2023-02-08 08:55:30,524 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208e0a819e1913c40aebbd0261f70920513 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208e0a819e1913c40aebbd0261f70920513 2023-02-08 08:55:30,524 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208547dd82594f94b97b01fec7e73f90b7e to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208547dd82594f94b97b01fec7e73f90b7e 2023-02-08 08:55:30,525 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230208845abc6337634d249733ad7bc14255fd to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230208845abc6337634d249733ad7bc14255fd 2023-02-08 08:55:30,525 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208e31e783a362b4ccdb2c77b63c75f96cc to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208e31e783a362b4ccdb2c77b63c75f96cc 2023-02-08 08:55:30,525 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208df405c3a445f4c1cad9d8184abd4dd96 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208df405c3a445f4c1cad9d8184abd4dd96 2023-02-08 08:55:30,531 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302088573e6ea6b2449f2b300daa6fa8fb08d to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302088573e6ea6b2449f2b300daa6fa8fb08d 2023-02-08 08:55:30,531 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-08 08:55:30,531 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208ee9b7e1c09a04572bfad5423c41f1272 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208ee9b7e1c09a04572bfad5423c41f1272 2023-02-08 08:55:30,531 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230208' 2023-02-08 08:55:30,531 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-08 08:55:30,531 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230208' 2023-02-08 08:55:30,531 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208f5ce04e42d824d54a5cf228d419280d6 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208f5ce04e42d824d54a5cf228d419280d6 2023-02-08 08:55:30,531 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-08 08:55:30,531 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230208' 2023-02-08 08:55:30,532 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 08:55:30,532 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 08:55:30,532 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 08:55:30,532 INFO [Listener at localhost/40763] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-02-08 08:55:30,532 INFO [Listener at localhost/40763] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 08:55:30,532 INFO [Listener at localhost/40763] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302080d8598f4b4cc48f1b6e3ccb5de6f7706_del] 2023-02-08 08:55:30,533 DEBUG [Listener at localhost/40763] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:30,534 DEBUG [Listener at localhost/40763] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302080d8598f4b4cc48f1b6e3ccb5de6f7706_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302080d8598f4b4cc48f1b6e3ccb5de6f7706_del 2023-02-08 08:55:30,534 INFO [Listener at localhost/40763] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-08 08:55:30,534 INFO [Listener at localhost/40763] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020897302e68d68d47de8bf71bae9199364b_del] 2023-02-08 08:55:30,534 DEBUG [Listener at localhost/40763] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:30,536 DEBUG [Listener at localhost/40763] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020897302e68d68d47de8bf71bae9199364b_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b2023020897302e68d68d47de8bf71bae9199364b_del 2023-02-08 08:55:30,536 INFO [Listener at localhost/40763] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-08 08:55:30,536 INFO [Listener at localhost/40763] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230208019610ae7936458a98ce9254ab0c3f84_del] 2023-02-08 08:55:30,536 DEBUG [Listener at localhost/40763] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:30,537 DEBUG [Listener at localhost/40763] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230208019610ae7936458a98ce9254ab0c3f84_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230208019610ae7936458a98ce9254ab0c3f84_del 2023-02-08 08:55:30,537 INFO [Listener at localhost/40763] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-08 08:55:30,540 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 08:55:30,543 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 08:55:30,546 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 08:55:30,547 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 08:55:30,551 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:30,551 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:30,552 INFO [Listener at localhost/40763] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-02-08 08:55:30,552 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-02-08 08:55:30,553 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-02-08 08:55:30,554 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:30,554 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:30,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-08 08:55:30,555 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846530555"}]},"ts":"1675846530555"} 2023-02-08 08:55:30,556 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-02-08 08:55:30,557 INFO [PEWorker-2] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-02-08 08:55:30,558 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=79e77e74725bcdadb1e041ea443ff410, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5f0bc0fd5600fb2bfd54b18349c52a4a, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=665fcb26db73b9d2ce507202c21c1ade, UNASSIGN}] 2023-02-08 08:55:30,559 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=665fcb26db73b9d2ce507202c21c1ade, UNASSIGN 2023-02-08 08:55:30,559 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5f0bc0fd5600fb2bfd54b18349c52a4a, UNASSIGN 2023-02-08 08:55:30,559 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=79e77e74725bcdadb1e041ea443ff410, UNASSIGN 2023-02-08 08:55:30,560 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=665fcb26db73b9d2ce507202c21c1ade, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:30,560 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=79e77e74725bcdadb1e041ea443ff410, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:30,560 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=5f0bc0fd5600fb2bfd54b18349c52a4a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:30,560 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675846530560"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846530560"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846530560"}]},"ts":"1675846530560"} 2023-02-08 08:55:30,560 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675846530560"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846530560"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846530560"}]},"ts":"1675846530560"} 2023-02-08 08:55:30,560 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675846530560"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846530560"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846530560"}]},"ts":"1675846530560"} 2023-02-08 08:55:30,561 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=91, state=RUNNABLE; CloseRegionProcedure 665fcb26db73b9d2ce507202c21c1ade, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:55:30,562 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=90, state=RUNNABLE; CloseRegionProcedure 5f0bc0fd5600fb2bfd54b18349c52a4a, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:55:30,562 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=89, state=RUNNABLE; CloseRegionProcedure 79e77e74725bcdadb1e041ea443ff410, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:55:30,655 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:30,655 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:30,656 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-08 08:55:30,713 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:55:30,713 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 5f0bc0fd5600fb2bfd54b18349c52a4a, disabling compactions & flushes 2023-02-08 08:55:30,713 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:30,713 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:30,714 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. after waiting 1 ms 2023-02-08 08:55:30,714 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:30,717 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-08 08:55:30,718 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:30,718 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 5f0bc0fd5600fb2bfd54b18349c52a4a: Waiting for close lock at 1675846530713Running coprocessor pre-close hooks at 1675846530713Disabling compacts and flushes for region at 1675846530713Disabling writes for close at 1675846530714 (+1 ms)Writing region close event to WAL at 1675846530715 (+1 ms)Running coprocessor post-close hooks at 1675846530718 (+3 ms)Closed at 1675846530718 2023-02-08 08:55:30,720 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:55:30,720 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:55:30,720 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 665fcb26db73b9d2ce507202c21c1ade, disabling compactions & flushes 2023-02-08 08:55:30,720 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:30,720 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:30,720 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. after waiting 0 ms 2023-02-08 08:55:30,720 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:30,720 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=5f0bc0fd5600fb2bfd54b18349c52a4a, regionState=CLOSED 2023-02-08 08:55:30,720 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675846530720"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846530720"}]},"ts":"1675846530720"} 2023-02-08 08:55:30,723 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=90 2023-02-08 08:55:30,723 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=90, state=SUCCESS; CloseRegionProcedure 5f0bc0fd5600fb2bfd54b18349c52a4a, server=jenkins-hbase4.apache.org,42239,1675846413348 in 159 msec 2023-02-08 08:55:30,724 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-08 08:55:30,724 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5f0bc0fd5600fb2bfd54b18349c52a4a, UNASSIGN in 165 msec 2023-02-08 08:55:30,725 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:30,725 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 665fcb26db73b9d2ce507202c21c1ade: Waiting for close lock at 1675846530720Running coprocessor pre-close hooks at 1675846530720Disabling compacts and flushes for region at 1675846530720Disabling writes for close at 1675846530720Writing region close event to WAL at 1675846530721 (+1 ms)Running coprocessor post-close hooks at 1675846530724 (+3 ms)Closed at 1675846530725 (+1 ms) 2023-02-08 08:55:30,726 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:55:30,726 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:55:30,726 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 79e77e74725bcdadb1e041ea443ff410, disabling compactions & flushes 2023-02-08 08:55:30,726 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:30,726 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:30,726 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. after waiting 0 ms 2023-02-08 08:55:30,726 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:30,726 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=665fcb26db73b9d2ce507202c21c1ade, regionState=CLOSED 2023-02-08 08:55:30,726 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675846530726"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846530726"}]},"ts":"1675846530726"} 2023-02-08 08:55:30,728 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=91 2023-02-08 08:55:30,728 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=91, state=SUCCESS; CloseRegionProcedure 665fcb26db73b9d2ce507202c21c1ade, server=jenkins-hbase4.apache.org,42239,1675846413348 in 166 msec 2023-02-08 08:55:30,729 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=665fcb26db73b9d2ce507202c21c1ade, UNASSIGN in 170 msec 2023-02-08 08:55:30,729 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-08 08:55:30,730 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:30,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 79e77e74725bcdadb1e041ea443ff410: Waiting for close lock at 1675846530726Running coprocessor pre-close hooks at 1675846530726Disabling compacts and flushes for region at 1675846530726Disabling writes for close at 1675846530726Writing region close event to WAL at 1675846530727 (+1 ms)Running coprocessor post-close hooks at 1675846530730 (+3 ms)Closed at 1675846530730 2023-02-08 08:55:30,731 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:55:30,731 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=79e77e74725bcdadb1e041ea443ff410, regionState=CLOSED 2023-02-08 08:55:30,732 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675846530731"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846530731"}]},"ts":"1675846530731"} 2023-02-08 08:55:30,733 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=89 2023-02-08 08:55:30,733 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=89, state=SUCCESS; CloseRegionProcedure 79e77e74725bcdadb1e041ea443ff410, server=jenkins-hbase4.apache.org,42239,1675846413348 in 170 msec 2023-02-08 08:55:30,734 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-08 08:55:30,734 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=79e77e74725bcdadb1e041ea443ff410, UNASSIGN in 175 msec 2023-02-08 08:55:30,735 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846530735"}]},"ts":"1675846530735"} 2023-02-08 08:55:30,736 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-02-08 08:55:30,737 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-02-08 08:55:30,738 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 185 msec 2023-02-08 08:55:30,856 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:30,856 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:30,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-08 08:55:30,857 INFO [Listener at localhost/40763] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-02-08 08:55:30,858 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:30,858 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:30,867 INFO [Listener at localhost/40763] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1675846517299-1675846530867 2023-02-08 08:55:30,867 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:30,867 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:30,868 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1675846517299-1675846530867 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-08 08:55:30,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1675846530868 (current time:1675846530868). 2023-02-08 08:55:30,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-02-08 08:55:30,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] zookeeper.ReadOnlyZKClient(139): Connect 0x7fae8f2c to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:55:30,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@17564b2a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:55:30,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:55:30,875 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60782, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:55:30,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7fae8f2c to 127.0.0.1:61594 2023-02-08 08:55:30,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:55:30,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-02-08 08:55:30,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1675846517299-1675846530867 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-08 08:55:30,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-08 08:55:30,879 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-08 08:55:30,880 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1675847130880 2023-02-08 08:55:30,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1675846517299-1675846530867 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-08 08:55:30,880 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1675846517299-1675846530867 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-02-08 08:55:30,880 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1675846517299-1675846530867 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-02-08 08:55:30,880 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-02-08 08:55:30,980 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 08:55:30,980 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:30,981 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:30,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675846517299-1675846530867 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 08:55:30,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1675846517299-1675846530867 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-08 08:55:30,981 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-02-08 08:55:31,182 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 08:55:31,182 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:31,182 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:31,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675846517299-1675846530867 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 08:55:31,182 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1675846517299-1675846530867 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-08 08:55:31,183 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-02-08 08:55:31,286 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1675846517299-1675846530867 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-02-08 08:55:31,287 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-08 08:55:31,287 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-08 08:55:31,287 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-08 08:55:31,287 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-08 08:55:31,287 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-08 08:55:31,287 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-08 08:55:31,287 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-08 08:55:31,288 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-08 08:55:31,290 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/26ec5f06dba04baab65d99a31660e218_SeqId_29_, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/cf8dba0dd3b944b8b254e36bb74fe986] hfiles 2023-02-08 08:55:31,290 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/26ec5f06dba04baab65d99a31660e218_SeqId_29_ 2023-02-08 08:55:31,290 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/cf8dba0dd3b944b8b254e36bb74fe986 2023-02-08 08:55:31,290 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/efae5049a5ce45f0b4753e1951139145, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/f3e93bc435324838999f46ea05b945c9_SeqId_29_] hfiles 2023-02-08 08:55:31,290 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/efae5049a5ce45f0b4753e1951139145 2023-02-08 08:55:31,290 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/f3e93bc435324838999f46ea05b945c9_SeqId_29_ 2023-02-08 08:55:31,290 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208b7e204bb7ae44f07a6af6c099bd6c36f, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230208fa2dcb1473ac4c99b6141964fea7b619, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302089e6aa4c566024b4ebf039193c398e1cb] hfiles 2023-02-08 08:55:31,290 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208b7e204bb7ae44f07a6af6c099bd6c36f 2023-02-08 08:55:31,290 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230208fa2dcb1473ac4c99b6141964fea7b619 2023-02-08 08:55:31,290 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302089e6aa4c566024b4ebf039193c398e1cb 2023-02-08 08:55:31,291 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/106075da0fe448b8905997054aa934fc, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/13de81ef75754182911e7c924c1662c1_SeqId_29_] hfiles 2023-02-08 08:55:31,291 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/106075da0fe448b8905997054aa934fc 2023-02-08 08:55:31,291 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/13de81ef75754182911e7c924c1662c1_SeqId_29_ 2023-02-08 08:55:31,291 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/e1af45530f26453281c5295607de8334] hfiles 2023-02-08 08:55:31,291 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/b24789fb11c54561aecf5266a1dab6c9] hfiles 2023-02-08 08:55:31,291 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/e1af45530f26453281c5295607de8334 2023-02-08 08:55:31,291 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/b24789fb11c54561aecf5266a1dab6c9 2023-02-08 08:55:31,291 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/03fe9ec355c64fee916a277921a47304] hfiles 2023-02-08 08:55:31,292 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/03fe9ec355c64fee916a277921a47304 2023-02-08 08:55:31,295 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208109b8e65fbf749c88b76ee104ef9f46e, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020820354c0d3bc849f09a31eec434c6b35d, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302086a575d9587e24ca2aadc4147fd222235, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208887265ec3b934b5fb6965e891b03f88f_del, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208a67f884cad3d40238bc523a59405b027, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302082c30bcfc25054141a3239a271f481236, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020847397442cb504fcf889973f84a22c5f9, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302086f950b436eda41f0962a5eccf190e571, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302087b26c8b1b47740e8b7fffa6d100ab412_del, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302088fe72b3b4a7f48a3a8a02594a7eba7e2, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302080420b0823ab2472fbd59ab7c177a0363, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020854c51eb8b80e424697b30e0801b27a40, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020870729d0a550e41bb8659552a71588f5f, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208909c1d78aa524f7bb29ed96ee3bd693c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208eadaa52fb7544e3798a0b7ce5b5d7579_del] hfiles 2023-02-08 08:55:31,295 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208109b8e65fbf749c88b76ee104ef9f46e 2023-02-08 08:55:31,295 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b2023020820354c0d3bc849f09a31eec434c6b35d 2023-02-08 08:55:31,295 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302086a575d9587e24ca2aadc4147fd222235 2023-02-08 08:55:31,295 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208887265ec3b934b5fb6965e891b03f88f_del 2023-02-08 08:55:31,295 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208a67f884cad3d40238bc523a59405b027 2023-02-08 08:55:31,295 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302082c30bcfc25054141a3239a271f481236 2023-02-08 08:55:31,295 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023020847397442cb504fcf889973f84a22c5f9 2023-02-08 08:55:31,295 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302086f950b436eda41f0962a5eccf190e571 2023-02-08 08:55:31,295 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302087b26c8b1b47740e8b7fffa6d100ab412_del 2023-02-08 08:55:31,296 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302088fe72b3b4a7f48a3a8a02594a7eba7e2 2023-02-08 08:55:31,296 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302080420b0823ab2472fbd59ab7c177a0363 2023-02-08 08:55:31,296 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020854c51eb8b80e424697b30e0801b27a40 2023-02-08 08:55:31,296 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e2023020870729d0a550e41bb8659552a71588f5f 2023-02-08 08:55:31,296 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208909c1d78aa524f7bb29ed96ee3bd693c 2023-02-08 08:55:31,296 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208eadaa52fb7544e3798a0b7ce5b5d7579_del 2023-02-08 08:55:31,483 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 08:55:31,483 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:31,483 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:31,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675846517299-1675846530867 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 08:55:31,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1675846517299-1675846530867 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-08 08:55:31,484 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-02-08 08:55:31,705 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1675846517299-1675846530867 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-02-08 08:55:31,706 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1675846517299-1675846530867 2023-02-08 08:55:31,706 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1675846517299-1675846530867 2023-02-08 08:55:31,984 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 08:55:31,984 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:31,984 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:31,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675846517299-1675846530867 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 08:55:31,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1675846517299-1675846530867 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-08 08:55:31,985 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-02-08 08:55:32,130 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1675846517299-1675846530867 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.hbase-snapshot/hbase-failsafe-snaptb-1675846517299-1675846530867 2023-02-08 08:55:32,944 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1675846517299-1675846530867 of table testCompactionWithHFileLink completed 2023-02-08 08:55:32,944 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1675846517299-1675846530867 2023-02-08 08:55:32,945 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1675846517299-1675846530867 2023-02-08 08:55:32,945 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1675846517299-1675846530867 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1675846530880Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1675846517299-1675846530867 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1675846531286 (+406 ms)Consolidate snapshot: hbase-failsafe-snaptb-1675846517299-1675846530867 at 1675846531706 (+420 ms)Loading Region manifests for hbase-failsafe-snaptb-1675846517299-1675846530867 at 1675846531706Writing data manifest for hbase-failsafe-snaptb-1675846517299-1675846530867 at 1675846531709 (+3 ms)Verifying snapshot: hbase-failsafe-snaptb-1675846517299-1675846530867 at 1675846532114 (+405 ms)Snapshot hbase-failsafe-snaptb-1675846517299-1675846530867 of table testCompactionWithHFileLink completed at 1675846532944 (+830 ms) 2023-02-08 08:55:32,946 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-08 08:55:32,947 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 2.0680 sec 2023-02-08 08:55:32,986 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-08 08:55:32,986 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:32,986 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:32,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1675846517299-1675846530867 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-08 08:55:32,987 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1675846517299-1675846530867 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-02-08 08:55:32,988 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:32,988 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:33,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1675846517299" table: "testCompactionWithHFileLink" creation_time: 1675846517309 type: FLUSH version: 2 ttl: 0 ) 2023-02-08 08:55:33,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1675846517299 as table=testCompactionWithHFileLink 2023-02-08 08:55:33,027 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:33,027 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:33,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-08 08:55:33,128 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:33,128 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:33,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-08 08:55:33,329 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:33,329 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:33,330 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-08 08:55:33,435 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-02-08 08:55:33,435 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-02-08 08:55:33,436 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-02-08 08:55:33,437 INFO [PEWorker-1] procedure.RestoreSnapshotProcedure(374): Starting restore snapshot={ ss=snaptb-1675846517299 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-08 08:55:33,441 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1675846517299" table: "testCompactionWithHFileLink" creation_time: 1675846517309 type: FLUSH version: 2 ttl: 0 2023-02-08 08:55:33,442 DEBUG [PEWorker-1] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink 2023-02-08 08:55:33,446 DEBUG [PEWorker-1] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-02-08 08:55:33,449 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230208fa2dcb1473ac4c99b6141964fea7b619 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230208fa2dcb1473ac4c99b6141964fea7b619 2023-02-08 08:55:33,450 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208b7e204bb7ae44f07a6af6c099bd6c36f to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208b7e204bb7ae44f07a6af6c099bd6c36f 2023-02-08 08:55:33,451 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302089e6aa4c566024b4ebf039193c398e1cb to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302089e6aa4c566024b4ebf039193c398e1cb 2023-02-08 08:55:33,451 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b2023020836ad795446ba45d68401de818d5a02f4 of snapshot=snaptb-1675846517299 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 08:55:33,455 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230208e0a819e1913c40aebbd0261f70920513 of snapshot=snaptb-1675846517299 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 08:55:33,458 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230208e31e783a362b4ccdb2c77b63c75f96cc of snapshot=snaptb-1675846517299 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 08:55:33,461 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230208ee9b7e1c09a04572bfad5423c41f1272 of snapshot=snaptb-1675846517299 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 08:55:33,464 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023020842137fd8be5a4f54a55e79a58a213e9a of snapshot=snaptb-1675846517299 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 08:55:33,467 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202302085696d16ea6fb44fe8532f61a0ed98c5c of snapshot=snaptb-1675846517299 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 08:55:33,470 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230208845abc6337634d249733ad7bc14255fd of snapshot=snaptb-1675846517299 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 08:55:33,473 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202302088573e6ea6b2449f2b300daa6fa8fb08d of snapshot=snaptb-1675846517299 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 08:55:33,476 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202302082721e52113d24baca886c5b893723eee of snapshot=snaptb-1675846517299 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 08:55:33,479 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230208547dd82594f94b97b01fec7e73f90b7e of snapshot=snaptb-1675846517299 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 08:55:33,482 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230208df405c3a445f4c1cad9d8184abd4dd96 of snapshot=snaptb-1675846517299 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 08:55:33,485 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230208f5ce04e42d824d54a5cf228d419280d6 of snapshot=snaptb-1675846517299 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-08 08:55:33,489 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208887265ec3b934b5fb6965e891b03f88f_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230208887265ec3b934b5fb6965e891b03f88f_del 2023-02-08 08:55:33,491 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302087b26c8b1b47740e8b7fffa6d100ab412_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302087b26c8b1b47740e8b7fffa6d100ab412_del 2023-02-08 08:55:33,492 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208eadaa52fb7544e3798a0b7ce5b5d7579_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230208eadaa52fb7544e3798a0b7ce5b5d7579_del 2023-02-08 08:55:33,492 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(223): region to restore: 5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:55:33,493 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(223): region to restore: 665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:55:33,493 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(223): region to restore: 79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:55:33,496 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/cf8dba0dd3b944b8b254e36bb74fe986 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/cf8dba0dd3b944b8b254e36bb74fe986 2023-02-08 08:55:33,496 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/106075da0fe448b8905997054aa934fc to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/106075da0fe448b8905997054aa934fc 2023-02-08 08:55:33,496 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/f3e93bc435324838999f46ea05b945c9_SeqId_29_ to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/f3e93bc435324838999f46ea05b945c9_SeqId_29_ 2023-02-08 08:55:33,497 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/26ec5f06dba04baab65d99a31660e218_SeqId_29_ to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/26ec5f06dba04baab65d99a31660e218_SeqId_29_ 2023-02-08 08:55:33,497 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 284ce0dd25ac49459e03d8d9b0666ef9 of snapshot=snaptb-1675846517299 to region=5f0bc0fd5600fb2bfd54b18349c52a4a table=testCompactionWithHFileLink 2023-02-08 08:55:33,497 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/13de81ef75754182911e7c924c1662c1_SeqId_29_ to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/13de81ef75754182911e7c924c1662c1_SeqId_29_ 2023-02-08 08:55:33,497 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/efae5049a5ce45f0b4753e1951139145 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/efae5049a5ce45f0b4753e1951139145 2023-02-08 08:55:33,497 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 7c18a1b1124b437aa6ac4ca4574bd6aa of snapshot=snaptb-1675846517299 to region=665fcb26db73b9d2ce507202c21c1ade table=testCompactionWithHFileLink 2023-02-08 08:55:33,497 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 0188e80890ce408c90da25548b3d2a57 of snapshot=snaptb-1675846517299 to region=79e77e74725bcdadb1e041ea443ff410 table=testCompactionWithHFileLink 2023-02-08 08:55:33,501 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink acb7ff0c34d84d7695b97d48daefdb93 of snapshot=snaptb-1675846517299 to region=79e77e74725bcdadb1e041ea443ff410 table=testCompactionWithHFileLink 2023-02-08 08:55:33,501 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 573bbd30fdb14700861a01a224793061 of snapshot=snaptb-1675846517299 to region=665fcb26db73b9d2ce507202c21c1ade table=testCompactionWithHFileLink 2023-02-08 08:55:33,501 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5477f14eaa574e989fe111de5de2a17f of snapshot=snaptb-1675846517299 to region=5f0bc0fd5600fb2bfd54b18349c52a4a table=testCompactionWithHFileLink 2023-02-08 08:55:33,504 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 7df916e5f1c94fa0a8259f67ef2aec54 of snapshot=snaptb-1675846517299 to region=79e77e74725bcdadb1e041ea443ff410 table=testCompactionWithHFileLink 2023-02-08 08:55:33,505 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d63b5936006140559ed94f3e9779142c of snapshot=snaptb-1675846517299 to region=665fcb26db73b9d2ce507202c21c1ade table=testCompactionWithHFileLink 2023-02-08 08:55:33,505 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink dea8e16f3dd74291809c3038eb6c855c of snapshot=snaptb-1675846517299 to region=5f0bc0fd5600fb2bfd54b18349c52a4a table=testCompactionWithHFileLink 2023-02-08 08:55:33,509 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink dc99f56e914e4cddbaa43f5992b35a20 of snapshot=snaptb-1675846517299 to region=79e77e74725bcdadb1e041ea443ff410 table=testCompactionWithHFileLink 2023-02-08 08:55:33,509 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 55af94394e4d4776a9a83012121fb840 of snapshot=snaptb-1675846517299 to region=5f0bc0fd5600fb2bfd54b18349c52a4a table=testCompactionWithHFileLink 2023-02-08 08:55:33,516 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 3a17e09e335446e1b6ed1f591fb25d82 of snapshot=snaptb-1675846517299 to region=665fcb26db73b9d2ce507202c21c1ade table=testCompactionWithHFileLink 2023-02-08 08:55:33,519 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/e1af45530f26453281c5295607de8334 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/e1af45530f26453281c5295607de8334 2023-02-08 08:55:33,520 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 8eee8a31f2ca450389d1a3ee3f1350b0 of snapshot=snaptb-1675846517299 to region=5f0bc0fd5600fb2bfd54b18349c52a4a table=testCompactionWithHFileLink 2023-02-08 08:55:33,520 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/03fe9ec355c64fee916a277921a47304 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/03fe9ec355c64fee916a277921a47304 2023-02-08 08:55:33,520 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 6980bf1172c44a8c8fb61eef66ebcc3e of snapshot=snaptb-1675846517299 to region=79e77e74725bcdadb1e041ea443ff410 table=testCompactionWithHFileLink 2023-02-08 08:55:33,521 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/b24789fb11c54561aecf5266a1dab6c9 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/b24789fb11c54561aecf5266a1dab6c9 2023-02-08 08:55:33,521 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 729ebf7c577943268499efb8cb0a07ba of snapshot=snaptb-1675846517299 to region=665fcb26db73b9d2ce507202c21c1ade table=testCompactionWithHFileLink 2023-02-08 08:55:33,523 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 60be5913ca21413f86163127d6eed588 of snapshot=snaptb-1675846517299 to region=79e77e74725bcdadb1e041ea443ff410 table=testCompactionWithHFileLink 2023-02-08 08:55:33,524 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink dc241889a15f41ffbbbba991bb167763 of snapshot=snaptb-1675846517299 to region=5f0bc0fd5600fb2bfd54b18349c52a4a table=testCompactionWithHFileLink 2023-02-08 08:55:33,525 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e5a1c334760a448d8a1d5fa71666c54c of snapshot=snaptb-1675846517299 to region=665fcb26db73b9d2ce507202c21c1ade table=testCompactionWithHFileLink 2023-02-08 08:55:33,527 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a50110e6550e41a68e419e663403be7b of snapshot=snaptb-1675846517299 to region=79e77e74725bcdadb1e041ea443ff410 table=testCompactionWithHFileLink 2023-02-08 08:55:33,527 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink eab5fe674d46495898c58cd9b8ca04bc of snapshot=snaptb-1675846517299 to region=5f0bc0fd5600fb2bfd54b18349c52a4a table=testCompactionWithHFileLink 2023-02-08 08:55:33,528 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 71e29344e20f4cd38dd724239183b532 of snapshot=snaptb-1675846517299 to region=665fcb26db73b9d2ce507202c21c1ade table=testCompactionWithHFileLink 2023-02-08 08:55:33,531 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d7411eeea5ed4ed2936ab18ceb564590 of snapshot=snaptb-1675846517299 to region=79e77e74725bcdadb1e041ea443ff410 table=testCompactionWithHFileLink 2023-02-08 08:55:33,531 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 12f2df8055fc4e4da85427eb05d0bbff of snapshot=snaptb-1675846517299 to region=5f0bc0fd5600fb2bfd54b18349c52a4a table=testCompactionWithHFileLink 2023-02-08 08:55:33,531 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 299eac1ba6e746f28065bcf304ba36fd of snapshot=snaptb-1675846517299 to region=665fcb26db73b9d2ce507202c21c1ade table=testCompactionWithHFileLink 2023-02-08 08:55:33,535 INFO [PEWorker-1] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1675846517299" table: "testCompactionWithHFileLink" creation_time: 1675846517309 type: FLUSH version: 2 ttl: 0 2023-02-08 08:55:33,537 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846533536"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:33,537 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846533536"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:33,537 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1675846533536"}]},"ts":"9223372036854775807"} 2023-02-08 08:55:33,538 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-08 08:55:33,539 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 5f0bc0fd5600fb2bfd54b18349c52a4a, NAME => 'testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => 665fcb26db73b9d2ce507202c21c1ade, NAME => 'testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 79e77e74725bcdadb1e041ea443ff410, NAME => 'testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410.', STARTKEY => '', ENDKEY => '1'}] 2023-02-08 08:55:33,539 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675846533537"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846533537"}]},"ts":"1675846533537"} 2023-02-08 08:55:33,539 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675846533537"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846533537"}]},"ts":"1675846533537"} 2023-02-08 08:55:33,539 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675846533537"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846533537"}]},"ts":"1675846533537"} 2023-02-08 08:55:33,540 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-08 08:55:33,540 INFO [PEWorker-1] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-02-08 08:55:33,540 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 5f0bc0fd5600fb2bfd54b18349c52a4a, NAME => 'testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => 665fcb26db73b9d2ce507202c21c1ade, NAME => 'testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 79e77e74725bcdadb1e041ea443ff410, NAME => 'testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410.', STARTKEY => '', ENDKEY => '1'}] 2023-02-08 08:55:33,541 INFO [PEWorker-1] procedure.RestoreSnapshotProcedure(447): Restore snapshot={ ss=snaptb-1675846517299 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-02-08 08:55:33,542 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1675846517299" table: "testCompactionWithHFileLink" creation_time: 1675846517309 type: FLUSH version: 2 ttl: 0 ) in 525 msec 2023-02-08 08:55:33,630 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:33,631 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:33,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-08 08:55:33,632 INFO [Listener at localhost/40763] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-02-08 08:55:33,632 INFO [Listener at localhost/40763] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1675846517299-1675846530867 2023-02-08 08:55:33,632 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:33,632 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:33,636 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(748): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1675846517299-1675846530867" 2023-02-08 08:55:33,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1675846517299-1675846530867 2023-02-08 08:55:33,640 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:33,640 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:33,640 INFO [Listener at localhost/40763] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-02-08 08:55:33,644 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$10(2372): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-02-08 08:55:33,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-02-08 08:55:33,652 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:33,652 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:33,653 INFO [PEWorker-5] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-02-08 08:55:33,653 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846533653"}]},"ts":"1675846533653"} 2023-02-08 08:55:33,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-08 08:55:33,654 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-02-08 08:55:33,656 DEBUG [PEWorker-5] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-02-08 08:55:33,656 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=79e77e74725bcdadb1e041ea443ff410, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5f0bc0fd5600fb2bfd54b18349c52a4a, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=665fcb26db73b9d2ce507202c21c1ade, ASSIGN}] 2023-02-08 08:55:33,657 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=665fcb26db73b9d2ce507202c21c1ade, ASSIGN 2023-02-08 08:55:33,657 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5f0bc0fd5600fb2bfd54b18349c52a4a, ASSIGN 2023-02-08 08:55:33,658 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=79e77e74725bcdadb1e041ea443ff410, ASSIGN 2023-02-08 08:55:33,658 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=665fcb26db73b9d2ce507202c21c1ade, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-08 08:55:33,658 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5f0bc0fd5600fb2bfd54b18349c52a4a, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-08 08:55:33,658 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=79e77e74725bcdadb1e041ea443ff410, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-08 08:55:33,753 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:33,753 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:33,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-08 08:55:33,810 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=5f0bc0fd5600fb2bfd54b18349c52a4a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:33,810 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=665fcb26db73b9d2ce507202c21c1ade, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:33,810 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=79e77e74725bcdadb1e041ea443ff410, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:33,810 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675846533809"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846533809"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846533809"}]},"ts":"1675846533809"} 2023-02-08 08:55:33,810 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675846533809"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846533809"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846533809"}]},"ts":"1675846533809"} 2023-02-08 08:55:33,810 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675846533809"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846533809"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846533809"}]},"ts":"1675846533809"} 2023-02-08 08:55:33,811 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=99, state=RUNNABLE; OpenRegionProcedure 5f0bc0fd5600fb2bfd54b18349c52a4a, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:55:33,812 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=100, state=RUNNABLE; OpenRegionProcedure 665fcb26db73b9d2ce507202c21c1ade, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:55:33,812 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=98, state=RUNNABLE; OpenRegionProcedure 79e77e74725bcdadb1e041ea443ff410, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:55:33,954 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:33,955 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:33,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-08 08:55:33,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:33,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 79e77e74725bcdadb1e041ea443ff410, NAME => 'testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410.', STARTKEY => '', ENDKEY => '1'} 2023-02-08 08:55:33,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:55:33,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:33,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:55:33,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:55:33,966 INFO [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:55:33,967 DEBUG [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1 2023-02-08 08:55:33,967 DEBUG [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1 2023-02-08 08:55:33,967 INFO [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 79e77e74725bcdadb1e041ea443ff410 columnFamilyName family1 2023-02-08 08:55:33,979 DEBUG [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-0188e80890ce408c90da25548b3d2a57 2023-02-08 08:55:33,984 DEBUG [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-7df916e5f1c94fa0a8259f67ef2aec54 2023-02-08 08:55:33,989 DEBUG [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-acb7ff0c34d84d7695b97d48daefdb93 2023-02-08 08:55:33,994 DEBUG [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-dc99f56e914e4cddbaa43f5992b35a20 2023-02-08 08:55:33,994 INFO [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] regionserver.HStore(310): Store=79e77e74725bcdadb1e041ea443ff410/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:33,994 INFO [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:55:33,995 DEBUG [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2 2023-02-08 08:55:33,995 DEBUG [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2 2023-02-08 08:55:33,995 INFO [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 79e77e74725bcdadb1e041ea443ff410 columnFamilyName family2 2023-02-08 08:55:34,002 DEBUG [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-60be5913ca21413f86163127d6eed588 2023-02-08 08:55:34,006 DEBUG [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-6980bf1172c44a8c8fb61eef66ebcc3e 2023-02-08 08:55:34,010 DEBUG [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-a50110e6550e41a68e419e663403be7b 2023-02-08 08:55:34,014 DEBUG [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-d7411eeea5ed4ed2936ab18ceb564590 2023-02-08 08:55:34,015 INFO [StoreOpener-79e77e74725bcdadb1e041ea443ff410-1] regionserver.HStore(310): Store=79e77e74725bcdadb1e041ea443ff410/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:34,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:55:34,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:55:34,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:55:34,018 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:55:34,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 79e77e74725bcdadb1e041ea443ff410; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72888402, jitterRate=0.0861218273639679}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:55:34,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 79e77e74725bcdadb1e041ea443ff410: Running coprocessor pre-open hook at 1675846533965Writing region info on filesystem at 1675846533965Initializing all the Stores at 1675846533966 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846533966Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846533966Cleaning up temporary data from old regions at 1675846534017 (+51 ms)Cleaning up detritus from prior splits at 1675846534017Running coprocessor post-open hooks at 1675846534019 (+2 ms)Region opened successfully at 1675846534019 2023-02-08 08:55:34,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410., pid=103, masterSystemTime=1675846533962 2023-02-08 08:55:34,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:55:34,020 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:55:34,020 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-08 08:55:34,020 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-08 08:55:34,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:34,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:34,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:34,021 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 665fcb26db73b9d2ce507202c21c1ade, NAME => 'testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade.', STARTKEY => '2', ENDKEY => ''} 2023-02-08 08:55:34,022 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=79e77e74725bcdadb1e041ea443ff410, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:34,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:55:34,022 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675846534022"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846534022"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846534022"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846534022"}]},"ts":"1675846534022"} 2023-02-08 08:55:34,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:34,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:55:34,022 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:55:34,023 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-08 08:55:34,023 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1912): 79e77e74725bcdadb1e041ea443ff410/family1 is initiating minor compaction (all files) 2023-02-08 08:55:34,023 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 79e77e74725bcdadb1e041ea443ff410/family1 in testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:34,023 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-0188e80890ce408c90da25548b3d2a57, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-acb7ff0c34d84d7695b97d48daefdb93, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-7df916e5f1c94fa0a8259f67ef2aec54, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-dc99f56e914e4cddbaa43f5992b35a20] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp, totalSize=21.6 K 2023-02-08 08:55:34,023 INFO [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:55:34,023 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-08 08:55:34,023 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.HStore(1912): 79e77e74725bcdadb1e041ea443ff410/family2 is initiating minor compaction (all files) 2023-02-08 08:55:34,023 INFO [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 79e77e74725bcdadb1e041ea443ff410/family2 in testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:55:34,024 INFO [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-6980bf1172c44a8c8fb61eef66ebcc3e, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-60be5913ca21413f86163127d6eed588, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-a50110e6550e41a68e419e663403be7b, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-d7411eeea5ed4ed2936ab18ceb564590] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp, totalSize=20.4 K 2023-02-08 08:55:34,024 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-0188e80890ce408c90da25548b3d2a57, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846510548 2023-02-08 08:55:34,024 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-6980bf1172c44a8c8fb61eef66ebcc3e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846510548 2023-02-08 08:55:34,024 DEBUG [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1 2023-02-08 08:55:34,024 DEBUG [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1 2023-02-08 08:55:34,024 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=98 2023-02-08 08:55:34,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=98, state=SUCCESS; OpenRegionProcedure 79e77e74725bcdadb1e041ea443ff410, server=jenkins-hbase4.apache.org,42239,1675846413348 in 211 msec 2023-02-08 08:55:34,025 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-acb7ff0c34d84d7695b97d48daefdb93, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846511935 2023-02-08 08:55:34,025 INFO [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 665fcb26db73b9d2ce507202c21c1ade columnFamilyName family1 2023-02-08 08:55:34,025 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=79e77e74725bcdadb1e041ea443ff410, ASSIGN in 369 msec 2023-02-08 08:55:34,025 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-60be5913ca21413f86163127d6eed588, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846511935 2023-02-08 08:55:34,026 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-7df916e5f1c94fa0a8259f67ef2aec54, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846513710 2023-02-08 08:55:34,026 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-a50110e6550e41a68e419e663403be7b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846513710 2023-02-08 08:55:34,027 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-dc99f56e914e4cddbaa43f5992b35a20, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846515501 2023-02-08 08:55:34,027 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-d7411eeea5ed4ed2936ab18ceb564590, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846515501 2023-02-08 08:55:34,038 DEBUG [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-3a17e09e335446e1b6ed1f591fb25d82 2023-02-08 08:55:34,046 DEBUG [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-573bbd30fdb14700861a01a224793061 2023-02-08 08:55:34,064 DEBUG [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-7c18a1b1124b437aa6ac4ca4574bd6aa 2023-02-08 08:55:34,080 INFO [RS:0;jenkins-hbase4:42239-longCompactions-0] throttle.PressureAwareThroughputController(145): 79e77e74725bcdadb1e041ea443ff410#family2#compaction#113 average throughput is 0.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 53.85 MB/second 2023-02-08 08:55:34,080 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] throttle.PressureAwareThroughputController(145): 79e77e74725bcdadb1e041ea443ff410#family1#compaction#114 average throughput is 0.39 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-08 08:55:34,080 DEBUG [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-d63b5936006140559ed94f3e9779142c 2023-02-08 08:55:34,081 INFO [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] regionserver.HStore(310): Store=665fcb26db73b9d2ce507202c21c1ade/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:34,081 INFO [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:55:34,082 DEBUG [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2 2023-02-08 08:55:34,082 DEBUG [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2 2023-02-08 08:55:34,082 INFO [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 665fcb26db73b9d2ce507202c21c1ade columnFamilyName family2 2023-02-08 08:55:34,099 DEBUG [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-299eac1ba6e746f28065bcf304ba36fd 2023-02-08 08:55:34,104 DEBUG [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-71e29344e20f4cd38dd724239183b532 2023-02-08 08:55:34,110 DEBUG [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-729ebf7c577943268499efb8cb0a07ba 2023-02-08 08:55:34,118 DEBUG [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-e5a1c334760a448d8a1d5fa71666c54c 2023-02-08 08:55:34,118 INFO [StoreOpener-665fcb26db73b9d2ce507202c21c1ade-1] regionserver.HStore(310): Store=665fcb26db73b9d2ce507202c21c1ade/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:34,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:55:34,120 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:55:34,121 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:55:34,122 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:55:34,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 665fcb26db73b9d2ce507202c21c1ade; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65804369, jitterRate=-0.0194384902715683}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:55:34,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 665fcb26db73b9d2ce507202c21c1ade: Running coprocessor pre-open hook at 1675846534022Writing region info on filesystem at 1675846534022Initializing all the Stores at 1675846534023 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846534023Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846534023Cleaning up temporary data from old regions at 1675846534120 (+97 ms)Cleaning up detritus from prior splits at 1675846534121 (+1 ms)Running coprocessor post-open hooks at 1675846534123 (+2 ms)Region opened successfully at 1675846534123 2023-02-08 08:55:34,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade., pid=102, masterSystemTime=1675846533962 2023-02-08 08:55:34,124 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-08 08:55:34,124 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-08 08:55:34,125 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:34,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:34,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:34,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5f0bc0fd5600fb2bfd54b18349c52a4a, NAME => 'testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a.', STARTKEY => '1', ENDKEY => '2'} 2023-02-08 08:55:34,126 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=665fcb26db73b9d2ce507202c21c1ade, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:34,126 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1675846534126"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846534126"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846534126"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846534126"}]},"ts":"1675846534126"} 2023-02-08 08:55:34,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:55:34,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:34,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:55:34,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:55:34,127 INFO [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:55:34,128 DEBUG [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1 2023-02-08 08:55:34,128 DEBUG [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1 2023-02-08 08:55:34,128 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=100 2023-02-08 08:55:34,128 INFO [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5f0bc0fd5600fb2bfd54b18349c52a4a columnFamilyName family1 2023-02-08 08:55:34,128 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=100, state=SUCCESS; OpenRegionProcedure 665fcb26db73b9d2ce507202c21c1ade, server=jenkins-hbase4.apache.org,42239,1675846413348 in 315 msec 2023-02-08 08:55:34,129 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=665fcb26db73b9d2ce507202c21c1ade, ASSIGN in 473 msec 2023-02-08 08:55:34,138 DEBUG [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-284ce0dd25ac49459e03d8d9b0666ef9 2023-02-08 08:55:34,143 DEBUG [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-5477f14eaa574e989fe111de5de2a17f 2023-02-08 08:55:34,151 DEBUG [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-55af94394e4d4776a9a83012121fb840 2023-02-08 08:55:34,158 DEBUG [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-dea8e16f3dd74291809c3038eb6c855c 2023-02-08 08:55:34,158 INFO [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] regionserver.HStore(310): Store=5f0bc0fd5600fb2bfd54b18349c52a4a/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:34,158 INFO [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:55:34,159 DEBUG [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2 2023-02-08 08:55:34,159 DEBUG [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2 2023-02-08 08:55:34,160 INFO [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5f0bc0fd5600fb2bfd54b18349c52a4a columnFamilyName family2 2023-02-08 08:55:34,168 DEBUG [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-12f2df8055fc4e4da85427eb05d0bbff 2023-02-08 08:55:34,174 DEBUG [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-8eee8a31f2ca450389d1a3ee3f1350b0 2023-02-08 08:55:34,179 DEBUG [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-dc241889a15f41ffbbbba991bb167763 2023-02-08 08:55:34,184 DEBUG [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] regionserver.HStore(539): loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-eab5fe674d46495898c58cd9b8ca04bc 2023-02-08 08:55:34,185 INFO [StoreOpener-5f0bc0fd5600fb2bfd54b18349c52a4a-1] regionserver.HStore(310): Store=5f0bc0fd5600fb2bfd54b18349c52a4a/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:34,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:55:34,186 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:55:34,187 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:55:34,188 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:55:34,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 5f0bc0fd5600fb2bfd54b18349c52a4a; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73653379, jitterRate=0.0975208729505539}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:55:34,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 5f0bc0fd5600fb2bfd54b18349c52a4a: Running coprocessor pre-open hook at 1675846534126Writing region info on filesystem at 1675846534126Initializing all the Stores at 1675846534127 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846534127Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846534127Cleaning up temporary data from old regions at 1675846534187 (+60 ms)Cleaning up detritus from prior splits at 1675846534187Running coprocessor post-open hooks at 1675846534189 (+2 ms)Region opened successfully at 1675846534189 2023-02-08 08:55:34,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a., pid=101, masterSystemTime=1675846533962 2023-02-08 08:55:34,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-08 08:55:34,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-08 08:55:34,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:34,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:34,191 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=5f0bc0fd5600fb2bfd54b18349c52a4a, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:34,191 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675846534191"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846534191"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846534191"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846534191"}]},"ts":"1675846534191"} 2023-02-08 08:55:34,194 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=99 2023-02-08 08:55:34,194 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=99, state=SUCCESS; OpenRegionProcedure 5f0bc0fd5600fb2bfd54b18349c52a4a, server=jenkins-hbase4.apache.org,42239,1675846413348 in 382 msec 2023-02-08 08:55:34,195 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=97 2023-02-08 08:55:34,195 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=5f0bc0fd5600fb2bfd54b18349c52a4a, ASSIGN in 538 msec 2023-02-08 08:55:34,195 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846534195"}]},"ts":"1675846534195"} 2023-02-08 08:55:34,196 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-02-08 08:55:34,197 INFO [PEWorker-2] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-02-08 08:55:34,199 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 552 msec 2023-02-08 08:55:34,256 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:34,256 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:34,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-08 08:55:34,257 INFO [Listener at localhost/40763] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-02-08 08:55:34,269 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 08:55:34,276 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 08:55:34,286 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 08:55:34,287 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 08:55:34,293 INFO [Listener at localhost/40763] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-08 08:55:34,307 INFO [Listener at localhost/40763] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-02-08 08:55:34,307 INFO [Listener at localhost/40763] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-02-08 08:55:34,307 DEBUG [Listener at localhost/40763] zookeeper.ReadOnlyZKClient(139): Connect 0x35721454 to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:55:34,312 DEBUG [Listener at localhost/40763] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@65f1e786, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:55:34,312 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 08:55:34,312 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 08:55:34,312 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 08:55:34,525 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.07692307692307693, tune throughput to 53.85 MB/second 2023-02-08 08:55:34,806 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208913197ba459d4b438f999e44a6da64e0 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230208913197ba459d4b438f999e44a6da64e0 2023-02-08 08:55:34,806 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 08:55:34,813 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208508015cdbad9487caa10f6ef6c357cc8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230208508015cdbad9487caa10f6ef6c357cc8 2023-02-08 08:55:34,814 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 08:55:34,820 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:55:34,823 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54966, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:55:34,830 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:34,830 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:34,831 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 08:55:34,834 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55634, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 08:55:34,835 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:34,835 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:34,849 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230208/family1/d145592d2f964ccaa2e04914be3a2f77 first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-02-08 08:55:34,850 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230208/family1/d145592d2f964ccaa2e04914be3a2f77}] 2023-02-08 08:55:34,850 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230208/family1/27bc01c403dd41b18b83c2bb3ab151fc first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-02-08 08:55:34,851 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230208/family1/27bc01c403dd41b18b83c2bb3ab151fc}] 2023-02-08 08:55:34,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230208/family1/27bc01c403dd41b18b83c2bb3ab151fc for inclusion in 665fcb26db73b9d2ce507202c21c1ade/family1 2023-02-08 08:55:34,864 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230208/family1/d145592d2f964ccaa2e04914be3a2f77 for inclusion in 5f0bc0fd5600fb2bfd54b18349c52a4a/family1 2023-02-08 08:55:34,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-02-08 08:55:34,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-02-08 08:55:34,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-08 08:55:34,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-08 08:55:34,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for 665fcb26db73b9d2ce507202c21c1ade: 2023-02-08 08:55:34,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for 5f0bc0fd5600fb2bfd54b18349c52a4a: 2023-02-08 08:55:34,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230208/family1/d145592d2f964ccaa2e04914be3a2f77 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__395s7ajuto4akc5lj742p9cdml7u09rk0c6eqsul8304r23h0mb5dqmktanr29ao/family1/d145592d2f964ccaa2e04914be3a2f77 2023-02-08 08:55:34,869 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230208/family1/27bc01c403dd41b18b83c2bb3ab151fc to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__j0b4fnneif81ljs4v9rouvnpm5kmbt635nnm8ksp0iap6rpkg2nju5ml7paf3epj/family1/27bc01c403dd41b18b83c2bb3ab151fc 2023-02-08 08:55:34,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__j0b4fnneif81ljs4v9rouvnpm5kmbt635nnm8ksp0iap6rpkg2nju5ml7paf3epj/family1/27bc01c403dd41b18b83c2bb3ab151fc as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/db809c82d4d84dadbf84f1b5d7e2a3d2_SeqId_34_ 2023-02-08 08:55:34,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__395s7ajuto4akc5lj742p9cdml7u09rk0c6eqsul8304r23h0mb5dqmktanr29ao/family1/d145592d2f964ccaa2e04914be3a2f77 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/b21a40c976d6492b80dffc0b17fd9c83_SeqId_34_ 2023-02-08 08:55:34,871 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__j0b4fnneif81ljs4v9rouvnpm5kmbt635nnm8ksp0iap6rpkg2nju5ml7paf3epj/family1/27bc01c403dd41b18b83c2bb3ab151fc into 665fcb26db73b9d2ce507202c21c1ade/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/db809c82d4d84dadbf84f1b5d7e2a3d2_SeqId_34_ - updating store file list. 2023-02-08 08:55:34,871 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__395s7ajuto4akc5lj742p9cdml7u09rk0c6eqsul8304r23h0mb5dqmktanr29ao/family1/d145592d2f964ccaa2e04914be3a2f77 into 5f0bc0fd5600fb2bfd54b18349c52a4a/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/b21a40c976d6492b80dffc0b17fd9c83_SeqId_34_ - updating store file list. 2023-02-08 08:55:34,874 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/db809c82d4d84dadbf84f1b5d7e2a3d2_SeqId_34_ into 665fcb26db73b9d2ce507202c21c1ade/family1 2023-02-08 08:55:34,875 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__j0b4fnneif81ljs4v9rouvnpm5kmbt635nnm8ksp0iap6rpkg2nju5ml7paf3epj/family1/27bc01c403dd41b18b83c2bb3ab151fc into 665fcb26db73b9d2ce507202c21c1ade/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/db809c82d4d84dadbf84f1b5d7e2a3d2_SeqId_34_) 2023-02-08 08:55:34,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__j0b4fnneif81ljs4v9rouvnpm5kmbt635nnm8ksp0iap6rpkg2nju5ml7paf3epj/family1/27bc01c403dd41b18b83c2bb3ab151fc 2023-02-08 08:55:34,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 08:55:34,875 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/b21a40c976d6492b80dffc0b17fd9c83_SeqId_34_ into 5f0bc0fd5600fb2bfd54b18349c52a4a/family1 2023-02-08 08:55:34,875 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__395s7ajuto4akc5lj742p9cdml7u09rk0c6eqsul8304r23h0mb5dqmktanr29ao/family1/d145592d2f964ccaa2e04914be3a2f77 into 5f0bc0fd5600fb2bfd54b18349c52a4a/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/b21a40c976d6492b80dffc0b17fd9c83_SeqId_34_) 2023-02-08 08:55:34,876 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__395s7ajuto4akc5lj742p9cdml7u09rk0c6eqsul8304r23h0mb5dqmktanr29ao/family1/d145592d2f964ccaa2e04914be3a2f77 2023-02-08 08:55:34,877 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-02-08 08:55:34,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(1912): 665fcb26db73b9d2ce507202c21c1ade/family1 is initiating minor compaction (all files) 2023-02-08 08:55:34,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@66f899eb; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-02-08 08:55:34,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 08:55:34,880 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 08:55:34,880 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:34,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-02-08 08:55:34,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(1912): 5f0bc0fd5600fb2bfd54b18349c52a4a/family1 is initiating minor compaction (all files) 2023-02-08 08:55:34,880 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@513e0526; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=6), splitQueue=0 2023-02-08 08:55:34,882 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023020842137fd8be5a4f54a55e79a58a213e9a to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023020842137fd8be5a4f54a55e79a58a213e9a 2023-02-08 08:55:34,882 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 08:55:34,883 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302085696d16ea6fb44fe8532f61a0ed98c5c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302085696d16ea6fb44fe8532f61a0ed98c5c 2023-02-08 08:55:34,883 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:34,884 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230208845abc6337634d249733ad7bc14255fd to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230208845abc6337634d249733ad7bc14255fd 2023-02-08 08:55:34,884 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023020836ad795446ba45d68401de818d5a02f4 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b2023020836ad795446ba45d68401de818d5a02f4 2023-02-08 08:55:34,885 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302088573e6ea6b2449f2b300daa6fa8fb08d to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302088573e6ea6b2449f2b300daa6fa8fb08d 2023-02-08 08:55:34,885 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-08 08:55:34,885 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230208' 2023-02-08 08:55:34,885 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230208e0a819e1913c40aebbd0261f70920513 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230208e0a819e1913c40aebbd0261f70920513 2023-02-08 08:55:34,886 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 08:55:34,886 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230208e31e783a362b4ccdb2c77b63c75f96cc to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230208e31e783a362b4ccdb2c77b63c75f96cc 2023-02-08 08:55:34,887 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230208ee9b7e1c09a04572bfad5423c41f1272 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230208ee9b7e1c09a04572bfad5423c41f1272 2023-02-08 08:55:34,887 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-08 08:55:34,887 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230208' 2023-02-08 08:55:34,888 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 08:55:35,205 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208b3394cadc82a48278a89e4b3bc86f0af to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230208b3394cadc82a48278a89e4b3bc86f0af 2023-02-08 08:55:35,206 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 08:55:35,223 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:35,223 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:35,230 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230208/family1/a711de0c6b0746999be45f7f03b65632 first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-02-08 08:55:35,230 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=32 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230208/family1/a711de0c6b0746999be45f7f03b65632}] 2023-02-08 08:55:35,232 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230208/family1/a711de0c6b0746999be45f7f03b65632 for inclusion in 79e77e74725bcdadb1e041ea443ff410/family1 2023-02-08 08:55:35,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-02-08 08:55:35,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last=1 2023-02-08 08:55:35,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for 79e77e74725bcdadb1e041ea443ff410: 2023-02-08 08:55:35,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230208/family1/a711de0c6b0746999be45f7f03b65632 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__dlg54kioh9mri4kp448camaj89i8rrimkn0bb09u08phaoe07df3oionsvogt35m/family1/a711de0c6b0746999be45f7f03b65632 2023-02-08 08:55:35,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__dlg54kioh9mri4kp448camaj89i8rrimkn0bb09u08phaoe07df3oionsvogt35m/family1/a711de0c6b0746999be45f7f03b65632 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/669529392c43466484e273eef4c73248_SeqId_34_ 2023-02-08 08:55:35,237 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__dlg54kioh9mri4kp448camaj89i8rrimkn0bb09u08phaoe07df3oionsvogt35m/family1/a711de0c6b0746999be45f7f03b65632 into 79e77e74725bcdadb1e041ea443ff410/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/669529392c43466484e273eef4c73248_SeqId_34_ - updating store file list. 2023-02-08 08:55:35,239 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/669529392c43466484e273eef4c73248_SeqId_34_ into 79e77e74725bcdadb1e041ea443ff410/family1 2023-02-08 08:55:35,239 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__dlg54kioh9mri4kp448camaj89i8rrimkn0bb09u08phaoe07df3oionsvogt35m/family1/a711de0c6b0746999be45f7f03b65632 into 79e77e74725bcdadb1e041ea443ff410/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/669529392c43466484e273eef4c73248_SeqId_34_) 2023-02-08 08:55:35,240 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testCompactionWithHFileLink__dlg54kioh9mri4kp448camaj89i8rrimkn0bb09u08phaoe07df3oionsvogt35m/family1/a711de0c6b0746999be45f7f03b65632 2023-02-08 08:55:35,241 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230208 table='testCompactionWithHFileLink' column='family1' 2023-02-08 08:55:35,242 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:35,243 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302082721e52113d24baca886c5b893723eee to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302082721e52113d24baca886c5b893723eee 2023-02-08 08:55:35,244 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230208547dd82594f94b97b01fec7e73f90b7e to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230208547dd82594f94b97b01fec7e73f90b7e 2023-02-08 08:55:35,245 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230208df405c3a445f4c1cad9d8184abd4dd96 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230208df405c3a445f4c1cad9d8184abd4dd96 2023-02-08 08:55:35,246 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230208f5ce04e42d824d54a5cf228d419280d6 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230208f5ce04e42d824d54a5cf228d419280d6 2023-02-08 08:55:35,246 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-08 08:55:35,246 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230208' 2023-02-08 08:55:35,246 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230208 for table='testCompactionWithHFileLink' and column='family1' 2023-02-08 08:55:35,246 INFO [Listener at localhost/40763] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-02-08 08:55:35,248 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 08:55:35,251 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 08:55:35,254 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 08:55:35,255 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 08:55:35,262 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 08:55:35,263 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 08:55:35,279 INFO [Listener at localhost/40763] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=285 (was 236) - Thread LEAK? -, OpenFileDescriptor=767 (was 601) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=136 (was 193), ProcessCount=168 (was 168), AvailableMemoryMB=4042 (was 4150) 2023-02-08 08:55:35,290 INFO [Listener at localhost/40763] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=285, OpenFileDescriptor=767, MaxFileDescriptor=60000, SystemLoadAverage=136, ProcessCount=168, AvailableMemoryMB=4041 2023-02-08 08:55:35,292 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:35,292 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:35,295 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$15(2997): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-02-08 08:55:35,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-02-08 08:55:35,301 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:35,301 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:35,301 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-02-08 08:55:35,303 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-08 08:55:35,304 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family1/dd3182bc965e462b916522e91e883f8f as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/dd3182bc965e462b916522e91e883f8f 2023-02-08 08:55:35,305 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 8 msec 2023-02-08 08:55:35,308 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/.tmp/family2/e48cfbfe1a86477d95baa56c9a3a380f as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/e48cfbfe1a86477d95baa56c9a3a380f 2023-02-08 08:55:35,309 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 79e77e74725bcdadb1e041ea443ff410/family1 of 79e77e74725bcdadb1e041ea443ff410 into dd3182bc965e462b916522e91e883f8f(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 08:55:35,309 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 79e77e74725bcdadb1e041ea443ff410: 2023-02-08 08:55:35,309 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410., storeName=79e77e74725bcdadb1e041ea443ff410/family1, priority=12, startTime=1675846534019; duration=1sec 2023-02-08 08:55:35,309 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=6), splitQueue=0 2023-02-08 08:55:35,309 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 665fcb26db73b9d2ce507202c21c1ade/family1 in testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:35,309 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-7c18a1b1124b437aa6ac4ca4574bd6aa, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-573bbd30fdb14700861a01a224793061, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-d63b5936006140559ed94f3e9779142c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-3a17e09e335446e1b6ed1f591fb25d82, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/db809c82d4d84dadbf84f1b5d7e2a3d2_SeqId_34_] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp, totalSize=28.9 K 2023-02-08 08:55:35,310 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-7c18a1b1124b437aa6ac4ca4574bd6aa, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846510550 2023-02-08 08:55:35,311 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-573bbd30fdb14700861a01a224793061, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846511936 2023-02-08 08:55:35,311 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-d63b5936006140559ed94f3e9779142c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846513711 2023-02-08 08:55:35,312 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-3a17e09e335446e1b6ed1f591fb25d82, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846515502 2023-02-08 08:55:35,312 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting db809c82d4d84dadbf84f1b5d7e2a3d2_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1675846510550 2023-02-08 08:55:35,313 INFO [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 79e77e74725bcdadb1e041ea443ff410/family2 of 79e77e74725bcdadb1e041ea443ff410 into e48cfbfe1a86477d95baa56c9a3a380f(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 08:55:35,313 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 79e77e74725bcdadb1e041ea443ff410: 2023-02-08 08:55:35,313 INFO [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410., storeName=79e77e74725bcdadb1e041ea443ff410/family2, priority=12, startTime=1675846534020; duration=1sec 2023-02-08 08:55:35,313 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=5), splitQueue=0 2023-02-08 08:55:35,313 INFO [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 5f0bc0fd5600fb2bfd54b18349c52a4a/family1 in testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:35,313 INFO [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-284ce0dd25ac49459e03d8d9b0666ef9, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-5477f14eaa574e989fe111de5de2a17f, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-dea8e16f3dd74291809c3038eb6c855c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-55af94394e4d4776a9a83012121fb840, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/b21a40c976d6492b80dffc0b17fd9c83_SeqId_34_] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp, totalSize=28.9 K 2023-02-08 08:55:35,313 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-0188e80890ce408c90da25548b3d2a57, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-acb7ff0c34d84d7695b97d48daefdb93, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-7df916e5f1c94fa0a8259f67ef2aec54, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-dc99f56e914e4cddbaa43f5992b35a20] to archive 2023-02-08 08:55:35,314 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:35,314 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-284ce0dd25ac49459e03d8d9b0666ef9, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846510549 2023-02-08 08:55:35,314 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-5477f14eaa574e989fe111de5de2a17f, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846511936 2023-02-08 08:55:35,315 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-dea8e16f3dd74291809c3038eb6c855c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846513711 2023-02-08 08:55:35,315 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-0188e80890ce408c90da25548b3d2a57 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-0188e80890ce408c90da25548b3d2a57 2023-02-08 08:55:35,316 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-55af94394e4d4776a9a83012121fb840, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846515501 2023-02-08 08:55:35,316 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] compactions.Compactor(207): Compacting b21a40c976d6492b80dffc0b17fd9c83_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1675846510549 2023-02-08 08:55:35,317 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-acb7ff0c34d84d7695b97d48daefdb93 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-acb7ff0c34d84d7695b97d48daefdb93 2023-02-08 08:55:35,318 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-7df916e5f1c94fa0a8259f67ef2aec54 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-7df916e5f1c94fa0a8259f67ef2aec54 2023-02-08 08:55:35,319 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-dc99f56e914e4cddbaa43f5992b35a20 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family1/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-dc99f56e914e4cddbaa43f5992b35a20 2023-02-08 08:55:35,321 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-6980bf1172c44a8c8fb61eef66ebcc3e, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-60be5913ca21413f86163127d6eed588, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-a50110e6550e41a68e419e663403be7b, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-d7411eeea5ed4ed2936ab18ceb564590] to archive 2023-02-08 08:55:35,321 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:35,323 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-6980bf1172c44a8c8fb61eef66ebcc3e to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-6980bf1172c44a8c8fb61eef66ebcc3e 2023-02-08 08:55:35,324 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-60be5913ca21413f86163127d6eed588 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-60be5913ca21413f86163127d6eed588 2023-02-08 08:55:35,326 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-a50110e6550e41a68e419e663403be7b to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-a50110e6550e41a68e419e663403be7b 2023-02-08 08:55:35,327 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-d7411eeea5ed4ed2936ab18ceb564590 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/family2/testCompactionWithHFileLink=79e77e74725bcdadb1e041ea443ff410-d7411eeea5ed4ed2936ab18ceb564590 2023-02-08 08:55:35,328 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] throttle.PressureAwareThroughputController(145): 665fcb26db73b9d2ce507202c21c1ade#family1#compaction#115 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-08 08:55:35,331 INFO [RS:0;jenkins-hbase4:42239-longCompactions-0] throttle.PressureAwareThroughputController(145): 5f0bc0fd5600fb2bfd54b18349c52a4a#family1#compaction#116 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-08 08:55:35,402 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:35,402 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:35,402 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-02-08 08:55:35,403 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:35,403 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:35,404 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-08 08:55:35,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-02-08 08:55:35,406 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:55:35,406 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-02-08 08:55:35,406 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:35,407 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:55:35,407 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:35,407 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-08 08:55:35,409 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02 2023-02-08 08:55:35,409 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8 2023-02-08 08:55:35,409 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f 2023-02-08 08:55:35,409 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02 empty. 2023-02-08 08:55:35,409 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f empty. 2023-02-08 08:55:35,409 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8 empty. 2023-02-08 08:55:35,409 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02 2023-02-08 08:55:35,409 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8 2023-02-08 08:55:35,410 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f 2023-02-08 08:55:35,410 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-02-08 08:55:35,507 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:35,508 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:35,508 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-08 08:55:35,709 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:35,709 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:35,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-08 08:55:35,817 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-02-08 08:55:35,819 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7671): creating {ENCODED => 7e8631296641f454e22df0ec18084b7f, NAME => 'ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp 2023-02-08 08:55:35,819 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7671): creating {ENCODED => f19baf8f1950f69f65e90c2ac697eb02, NAME => 'ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp 2023-02-08 08:55:35,819 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7671): creating {ENCODED => c4e6f28a0b52d9de77dc2ced57e02cc8, NAME => 'ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp 2023-02-08 08:55:36,010 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:36,010 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:36,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-08 08:55:36,226 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:36,226 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:36,226 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1603): Closing 7e8631296641f454e22df0ec18084b7f, disabling compactions & flushes 2023-02-08 08:55:36,226 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1603): Closing f19baf8f1950f69f65e90c2ac697eb02, disabling compactions & flushes 2023-02-08 08:55:36,226 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:55:36,226 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:36,226 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:55:36,226 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:55:36,226 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. after waiting 0 ms 2023-02-08 08:55:36,226 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1603): Closing c4e6f28a0b52d9de77dc2ced57e02cc8, disabling compactions & flushes 2023-02-08 08:55:36,226 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:55:36,226 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:55:36,226 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1837): Closed ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:55:36,226 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. after waiting 0 ms 2023-02-08 08:55:36,226 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:55:36,226 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:55:36,227 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:55:36,227 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:55:36,227 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1557): Region close journal for f19baf8f1950f69f65e90c2ac697eb02: Waiting for close lock at 1675846536226Disabling compacts and flushes for region at 1675846536226Disabling writes for close at 1675846536226Writing region close event to WAL at 1675846536226Closed at 1675846536226 2023-02-08 08:55:36,227 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. after waiting 0 ms 2023-02-08 08:55:36,226 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1557): Region close journal for 7e8631296641f454e22df0ec18084b7f: Waiting for close lock at 1675846536226Disabling compacts and flushes for region at 1675846536226Disabling writes for close at 1675846536226Writing region close event to WAL at 1675846536226Closed at 1675846536226 2023-02-08 08:55:36,227 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:55:36,227 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1837): Closed ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:55:36,227 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1557): Region close journal for c4e6f28a0b52d9de77dc2ced57e02cc8: Waiting for close lock at 1675846536226Disabling compacts and flushes for region at 1675846536226Disabling writes for close at 1675846536227 (+1 ms)Writing region close event to WAL at 1675846536227Closed at 1675846536227 2023-02-08 08:55:36,229 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:55:36,229 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675846536229"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846536229"}]},"ts":"1675846536229"} 2023-02-08 08:55:36,230 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675846536229"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846536229"}]},"ts":"1675846536229"} 2023-02-08 08:55:36,230 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1675846536229"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846536229"}]},"ts":"1675846536229"} 2023-02-08 08:55:36,231 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-08 08:55:36,232 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:55:36,232 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846536232"}]},"ts":"1675846536232"} 2023-02-08 08:55:36,233 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-02-08 08:55:36,236 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=f19baf8f1950f69f65e90c2ac697eb02, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c4e6f28a0b52d9de77dc2ced57e02cc8, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=7e8631296641f454e22df0ec18084b7f, ASSIGN}] 2023-02-08 08:55:36,238 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=f19baf8f1950f69f65e90c2ac697eb02, ASSIGN 2023-02-08 08:55:36,238 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c4e6f28a0b52d9de77dc2ced57e02cc8, ASSIGN 2023-02-08 08:55:36,238 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=7e8631296641f454e22df0ec18084b7f, ASSIGN 2023-02-08 08:55:36,238 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=f19baf8f1950f69f65e90c2ac697eb02, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=false 2023-02-08 08:55:36,238 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=7e8631296641f454e22df0ec18084b7f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=false 2023-02-08 08:55:36,238 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c4e6f28a0b52d9de77dc2ced57e02cc8, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=false 2023-02-08 08:55:36,270 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 08:55:36,391 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=7e8631296641f454e22df0ec18084b7f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:36,391 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=c4e6f28a0b52d9de77dc2ced57e02cc8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:36,391 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=f19baf8f1950f69f65e90c2ac697eb02, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:36,391 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675846536391"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846536391"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846536391"}]},"ts":"1675846536391"} 2023-02-08 08:55:36,391 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675846536391"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846536391"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846536391"}]},"ts":"1675846536391"} 2023-02-08 08:55:36,391 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1675846536391"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846536391"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846536391"}]},"ts":"1675846536391"} 2023-02-08 08:55:36,393 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=108, state=RUNNABLE; OpenRegionProcedure 7e8631296641f454e22df0ec18084b7f, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:55:36,393 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=107, state=RUNNABLE; OpenRegionProcedure c4e6f28a0b52d9de77dc2ced57e02cc8, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:55:36,394 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=106, state=RUNNABLE; OpenRegionProcedure f19baf8f1950f69f65e90c2ac697eb02, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:55:36,511 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:36,512 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:36,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-08 08:55:36,546 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family1/9e4e39a32e4944ceacbec902e7ba746c as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/9e4e39a32e4944ceacbec902e7ba746c 2023-02-08 08:55:36,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:55:36,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7e8631296641f454e22df0ec18084b7f, NAME => 'ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f.', STARTKEY => '2', ENDKEY => ''} 2023-02-08 08:55:36,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 7e8631296641f454e22df0ec18084b7f 2023-02-08 08:55:36,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:36,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7e8631296641f454e22df0ec18084b7f 2023-02-08 08:55:36,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7e8631296641f454e22df0ec18084b7f 2023-02-08 08:55:36,548 INFO [StoreOpener-7e8631296641f454e22df0ec18084b7f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 7e8631296641f454e22df0ec18084b7f 2023-02-08 08:55:36,550 DEBUG [StoreOpener-7e8631296641f454e22df0ec18084b7f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1 2023-02-08 08:55:36,550 DEBUG [StoreOpener-7e8631296641f454e22df0ec18084b7f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1 2023-02-08 08:55:36,550 INFO [StoreOpener-7e8631296641f454e22df0ec18084b7f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7e8631296641f454e22df0ec18084b7f columnFamilyName family1 2023-02-08 08:55:36,551 INFO [StoreOpener-7e8631296641f454e22df0ec18084b7f-1] regionserver.HStore(310): Store=7e8631296641f454e22df0ec18084b7f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:36,552 INFO [StoreOpener-7e8631296641f454e22df0ec18084b7f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 7e8631296641f454e22df0ec18084b7f 2023-02-08 08:55:36,552 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family1/a376cfe1c9ee4d4c81d3f7e030d189f4 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/a376cfe1c9ee4d4c81d3f7e030d189f4 2023-02-08 08:55:36,552 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 665fcb26db73b9d2ce507202c21c1ade/family1 of 665fcb26db73b9d2ce507202c21c1ade into 9e4e39a32e4944ceacbec902e7ba746c(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 08:55:36,552 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 665fcb26db73b9d2ce507202c21c1ade: 2023-02-08 08:55:36,552 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1675846534878; duration=1sec 2023-02-08 08:55:36,552 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-08 08:55:36,552 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-08 08:55:36,553 DEBUG [StoreOpener-7e8631296641f454e22df0ec18084b7f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2 2023-02-08 08:55:36,553 DEBUG [StoreOpener-7e8631296641f454e22df0ec18084b7f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2 2023-02-08 08:55:36,554 INFO [StoreOpener-7e8631296641f454e22df0ec18084b7f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7e8631296641f454e22df0ec18084b7f columnFamilyName family2 2023-02-08 08:55:36,555 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-08 08:55:36,555 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1912): 665fcb26db73b9d2ce507202c21c1ade/family2 is initiating minor compaction (all files) 2023-02-08 08:55:36,555 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 665fcb26db73b9d2ce507202c21c1ade/family2 in testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:55:36,555 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-729ebf7c577943268499efb8cb0a07ba, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-e5a1c334760a448d8a1d5fa71666c54c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-71e29344e20f4cd38dd724239183b532, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-299eac1ba6e746f28065bcf304ba36fd] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp, totalSize=20.4 K 2023-02-08 08:55:36,555 INFO [StoreOpener-7e8631296641f454e22df0ec18084b7f-1] regionserver.HStore(310): Store=7e8631296641f454e22df0ec18084b7f/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:36,556 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-729ebf7c577943268499efb8cb0a07ba, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846510550 2023-02-08 08:55:36,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f 2023-02-08 08:55:36,556 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f 2023-02-08 08:55:36,556 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-e5a1c334760a448d8a1d5fa71666c54c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846511936 2023-02-08 08:55:36,557 INFO [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 5f0bc0fd5600fb2bfd54b18349c52a4a/family1 of 5f0bc0fd5600fb2bfd54b18349c52a4a into a376cfe1c9ee4d4c81d3f7e030d189f4(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 08:55:36,557 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5f0bc0fd5600fb2bfd54b18349c52a4a: 2023-02-08 08:55:36,557 INFO [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1675846534880; duration=1sec 2023-02-08 08:55:36,557 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-08 08:55:36,557 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-08 08:55:36,558 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-71e29344e20f4cd38dd724239183b532, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846513711 2023-02-08 08:55:36,558 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-299eac1ba6e746f28065bcf304ba36fd, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846515502 2023-02-08 08:55:36,558 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:55:36,559 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-08 08:55:36,559 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.HStore(1912): 5f0bc0fd5600fb2bfd54b18349c52a4a/family2 is initiating minor compaction (all files) 2023-02-08 08:55:36,559 INFO [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 5f0bc0fd5600fb2bfd54b18349c52a4a/family2 in testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:55:36,559 INFO [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-8eee8a31f2ca450389d1a3ee3f1350b0, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-dc241889a15f41ffbbbba991bb167763, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-eab5fe674d46495898c58cd9b8ca04bc, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-12f2df8055fc4e4da85427eb05d0bbff] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp, totalSize=20.4 K 2023-02-08 08:55:36,560 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7e8631296641f454e22df0ec18084b7f 2023-02-08 08:55:36,560 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-8eee8a31f2ca450389d1a3ee3f1350b0, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846510549 2023-02-08 08:55:36,561 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-dc241889a15f41ffbbbba991bb167763, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846511936 2023-02-08 08:55:36,561 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:55:36,562 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-eab5fe674d46495898c58cd9b8ca04bc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846513711 2023-02-08 08:55:36,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7e8631296641f454e22df0ec18084b7f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59086730, jitterRate=-0.11953911185264587}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:55:36,562 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7e8631296641f454e22df0ec18084b7f: Running coprocessor pre-open hook at 1675846536547Writing region info on filesystem at 1675846536547Initializing all the Stores at 1675846536548 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846536548Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846536548Cleaning up temporary data from old regions at 1675846536557 (+9 ms)Cleaning up detritus from prior splits at 1675846536558 (+1 ms)Running coprocessor post-open hooks at 1675846536562 (+4 ms)Region opened successfully at 1675846536562 2023-02-08 08:55:36,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f., pid=109, masterSystemTime=1675846536544 2023-02-08 08:55:36,563 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-12f2df8055fc4e4da85427eb05d0bbff, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846515501 2023-02-08 08:55:36,564 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:55:36,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:55:36,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:55:36,564 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f19baf8f1950f69f65e90c2ac697eb02, NAME => 'ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02.', STARTKEY => '', ENDKEY => '1'} 2023-02-08 08:55:36,564 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=7e8631296641f454e22df0ec18084b7f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:36,564 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction f19baf8f1950f69f65e90c2ac697eb02 2023-02-08 08:55:36,565 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675846536564"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846536564"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846536564"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846536564"}]},"ts":"1675846536564"} 2023-02-08 08:55:36,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:36,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f19baf8f1950f69f65e90c2ac697eb02 2023-02-08 08:55:36,565 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f19baf8f1950f69f65e90c2ac697eb02 2023-02-08 08:55:36,567 INFO [StoreOpener-f19baf8f1950f69f65e90c2ac697eb02-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region f19baf8f1950f69f65e90c2ac697eb02 2023-02-08 08:55:36,568 DEBUG [StoreOpener-f19baf8f1950f69f65e90c2ac697eb02-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1 2023-02-08 08:55:36,568 DEBUG [StoreOpener-f19baf8f1950f69f65e90c2ac697eb02-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1 2023-02-08 08:55:36,569 INFO [StoreOpener-f19baf8f1950f69f65e90c2ac697eb02-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f19baf8f1950f69f65e90c2ac697eb02 columnFamilyName family1 2023-02-08 08:55:36,570 INFO [StoreOpener-f19baf8f1950f69f65e90c2ac697eb02-1] regionserver.HStore(310): Store=f19baf8f1950f69f65e90c2ac697eb02/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:36,570 INFO [StoreOpener-f19baf8f1950f69f65e90c2ac697eb02-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region f19baf8f1950f69f65e90c2ac697eb02 2023-02-08 08:55:36,570 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=108 2023-02-08 08:55:36,570 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=108, state=SUCCESS; OpenRegionProcedure 7e8631296641f454e22df0ec18084b7f, server=jenkins-hbase4.apache.org,42239,1675846413348 in 176 msec 2023-02-08 08:55:36,571 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=7e8631296641f454e22df0ec18084b7f, ASSIGN in 334 msec 2023-02-08 08:55:36,571 DEBUG [StoreOpener-f19baf8f1950f69f65e90c2ac697eb02-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2 2023-02-08 08:55:36,571 DEBUG [StoreOpener-f19baf8f1950f69f65e90c2ac697eb02-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2 2023-02-08 08:55:36,572 INFO [StoreOpener-f19baf8f1950f69f65e90c2ac697eb02-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f19baf8f1950f69f65e90c2ac697eb02 columnFamilyName family2 2023-02-08 08:55:36,573 INFO [StoreOpener-f19baf8f1950f69f65e90c2ac697eb02-1] regionserver.HStore(310): Store=f19baf8f1950f69f65e90c2ac697eb02/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:36,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02 2023-02-08 08:55:36,574 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] throttle.PressureAwareThroughputController(145): 665fcb26db73b9d2ce507202c21c1ade#family2#compaction#117 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-08 08:55:36,574 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02 2023-02-08 08:55:36,576 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:55:36,579 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f19baf8f1950f69f65e90c2ac697eb02 2023-02-08 08:55:36,581 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:55:36,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f19baf8f1950f69f65e90c2ac697eb02; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61054215, jitterRate=-0.09022130072116852}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:55:36,581 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f19baf8f1950f69f65e90c2ac697eb02: Running coprocessor pre-open hook at 1675846536565Writing region info on filesystem at 1675846536565Initializing all the Stores at 1675846536566 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846536566Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846536566Cleaning up temporary data from old regions at 1675846536575 (+9 ms)Cleaning up detritus from prior splits at 1675846536575Running coprocessor post-open hooks at 1675846536581 (+6 ms)Region opened successfully at 1675846536581 2023-02-08 08:55:36,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02., pid=111, masterSystemTime=1675846536544 2023-02-08 08:55:36,582 INFO [RS:0;jenkins-hbase4:42239-longCompactions-0] throttle.PressureAwareThroughputController(145): 5f0bc0fd5600fb2bfd54b18349c52a4a#family2#compaction#118 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-08 08:55:36,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:55:36,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:55:36,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:55:36,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c4e6f28a0b52d9de77dc2ced57e02cc8, NAME => 'ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8.', STARTKEY => '1', ENDKEY => '2'} 2023-02-08 08:55:36,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction c4e6f28a0b52d9de77dc2ced57e02cc8 2023-02-08 08:55:36,583 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=f19baf8f1950f69f65e90c2ac697eb02, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:36,584 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1675846536583"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846536583"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846536583"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846536583"}]},"ts":"1675846536583"} 2023-02-08 08:55:36,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:36,584 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c4e6f28a0b52d9de77dc2ced57e02cc8 2023-02-08 08:55:36,584 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c4e6f28a0b52d9de77dc2ced57e02cc8 2023-02-08 08:55:36,586 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=106 2023-02-08 08:55:36,586 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=106, state=SUCCESS; OpenRegionProcedure f19baf8f1950f69f65e90c2ac697eb02, server=jenkins-hbase4.apache.org,42239,1675846413348 in 192 msec 2023-02-08 08:55:36,587 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=f19baf8f1950f69f65e90c2ac697eb02, ASSIGN in 350 msec 2023-02-08 08:55:36,590 INFO [StoreOpener-c4e6f28a0b52d9de77dc2ced57e02cc8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c4e6f28a0b52d9de77dc2ced57e02cc8 2023-02-08 08:55:36,592 DEBUG [StoreOpener-c4e6f28a0b52d9de77dc2ced57e02cc8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1 2023-02-08 08:55:36,592 DEBUG [StoreOpener-c4e6f28a0b52d9de77dc2ced57e02cc8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1 2023-02-08 08:55:36,592 INFO [StoreOpener-c4e6f28a0b52d9de77dc2ced57e02cc8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c4e6f28a0b52d9de77dc2ced57e02cc8 columnFamilyName family1 2023-02-08 08:55:36,593 INFO [StoreOpener-c4e6f28a0b52d9de77dc2ced57e02cc8-1] regionserver.HStore(310): Store=c4e6f28a0b52d9de77dc2ced57e02cc8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:36,593 INFO [StoreOpener-c4e6f28a0b52d9de77dc2ced57e02cc8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region c4e6f28a0b52d9de77dc2ced57e02cc8 2023-02-08 08:55:36,594 DEBUG [StoreOpener-c4e6f28a0b52d9de77dc2ced57e02cc8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2 2023-02-08 08:55:36,594 DEBUG [StoreOpener-c4e6f28a0b52d9de77dc2ced57e02cc8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2 2023-02-08 08:55:36,594 INFO [StoreOpener-c4e6f28a0b52d9de77dc2ced57e02cc8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c4e6f28a0b52d9de77dc2ced57e02cc8 columnFamilyName family2 2023-02-08 08:55:36,595 INFO [StoreOpener-c4e6f28a0b52d9de77dc2ced57e02cc8-1] regionserver.HStore(310): Store=c4e6f28a0b52d9de77dc2ced57e02cc8/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:36,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8 2023-02-08 08:55:36,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8 2023-02-08 08:55:36,600 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:55:36,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c4e6f28a0b52d9de77dc2ced57e02cc8 2023-02-08 08:55:36,602 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:55:36,603 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/.tmp/family2/33ce95edc43741af86d4081a534a760b as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/33ce95edc43741af86d4081a534a760b 2023-02-08 08:55:36,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c4e6f28a0b52d9de77dc2ced57e02cc8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71757840, jitterRate=0.0692751407623291}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:55:36,603 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c4e6f28a0b52d9de77dc2ced57e02cc8: Running coprocessor pre-open hook at 1675846536584Writing region info on filesystem at 1675846536584Initializing all the Stores at 1675846536585 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846536585Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846536589 (+4 ms)Cleaning up temporary data from old regions at 1675846536599 (+10 ms)Cleaning up detritus from prior splits at 1675846536599Running coprocessor post-open hooks at 1675846536603 (+4 ms)Region opened successfully at 1675846536603 2023-02-08 08:55:36,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8., pid=110, masterSystemTime=1675846536544 2023-02-08 08:55:36,605 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:55:36,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:55:36,605 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=c4e6f28a0b52d9de77dc2ced57e02cc8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:36,605 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1675846536605"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846536605"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846536605"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846536605"}]},"ts":"1675846536605"} 2023-02-08 08:55:36,607 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 665fcb26db73b9d2ce507202c21c1ade/family2 of 665fcb26db73b9d2ce507202c21c1ade into 33ce95edc43741af86d4081a534a760b(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:55:36,607 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 665fcb26db73b9d2ce507202c21c1ade: 2023-02-08 08:55:36,607 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade., storeName=665fcb26db73b9d2ce507202c21c1ade/family2, priority=12, startTime=1675846534124; duration=0sec 2023-02-08 08:55:36,607 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-08 08:55:36,607 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-08 08:55:36,607 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=107 2023-02-08 08:55:36,607 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=107, state=SUCCESS; OpenRegionProcedure c4e6f28a0b52d9de77dc2ced57e02cc8, server=jenkins-hbase4.apache.org,42239,1675846413348 in 213 msec 2023-02-08 08:55:36,607 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-08 08:55:36,607 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-08 08:55:36,607 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. because compaction request was cancelled 2023-02-08 08:55:36,608 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-08 08:55:36,608 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-08 08:55:36,608 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-08 08:55:36,608 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. because compaction request was cancelled 2023-02-08 08:55:36,608 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=105 2023-02-08 08:55:36,609 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c4e6f28a0b52d9de77dc2ced57e02cc8, ASSIGN in 371 msec 2023-02-08 08:55:36,609 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:55:36,609 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846536609"}]},"ts":"1675846536609"} 2023-02-08 08:55:36,610 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-02-08 08:55:36,612 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:55:36,613 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.2080 sec 2023-02-08 08:55:36,618 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-7c18a1b1124b437aa6ac4ca4574bd6aa, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-573bbd30fdb14700861a01a224793061, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-d63b5936006140559ed94f3e9779142c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-3a17e09e335446e1b6ed1f591fb25d82, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/db809c82d4d84dadbf84f1b5d7e2a3d2_SeqId_34_] to archive 2023-02-08 08:55:36,618 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:36,619 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-7c18a1b1124b437aa6ac4ca4574bd6aa to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-7c18a1b1124b437aa6ac4ca4574bd6aa 2023-02-08 08:55:36,620 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-573bbd30fdb14700861a01a224793061 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-573bbd30fdb14700861a01a224793061 2023-02-08 08:55:36,621 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-d63b5936006140559ed94f3e9779142c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-d63b5936006140559ed94f3e9779142c 2023-02-08 08:55:36,622 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-3a17e09e335446e1b6ed1f591fb25d82 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-3a17e09e335446e1b6ed1f591fb25d82 2023-02-08 08:55:36,622 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/db809c82d4d84dadbf84f1b5d7e2a3d2_SeqId_34_ to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family1/db809c82d4d84dadbf84f1b5d7e2a3d2_SeqId_34_ 2023-02-08 08:55:36,624 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-729ebf7c577943268499efb8cb0a07ba, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-e5a1c334760a448d8a1d5fa71666c54c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-71e29344e20f4cd38dd724239183b532, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-299eac1ba6e746f28065bcf304ba36fd] to archive 2023-02-08 08:55:36,624 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:36,625 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-729ebf7c577943268499efb8cb0a07ba to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-729ebf7c577943268499efb8cb0a07ba 2023-02-08 08:55:36,626 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-e5a1c334760a448d8a1d5fa71666c54c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-e5a1c334760a448d8a1d5fa71666c54c 2023-02-08 08:55:36,626 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-71e29344e20f4cd38dd724239183b532 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-71e29344e20f4cd38dd724239183b532 2023-02-08 08:55:36,627 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-299eac1ba6e746f28065bcf304ba36fd to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/family2/testCompactionWithHFileLink=665fcb26db73b9d2ce507202c21c1ade-299eac1ba6e746f28065bcf304ba36fd 2023-02-08 08:55:36,628 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-284ce0dd25ac49459e03d8d9b0666ef9, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-5477f14eaa574e989fe111de5de2a17f, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-dea8e16f3dd74291809c3038eb6c855c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-55af94394e4d4776a9a83012121fb840, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/b21a40c976d6492b80dffc0b17fd9c83_SeqId_34_] to archive 2023-02-08 08:55:36,629 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:36,630 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-284ce0dd25ac49459e03d8d9b0666ef9 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-284ce0dd25ac49459e03d8d9b0666ef9 2023-02-08 08:55:36,631 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-5477f14eaa574e989fe111de5de2a17f to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-5477f14eaa574e989fe111de5de2a17f 2023-02-08 08:55:36,631 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-dea8e16f3dd74291809c3038eb6c855c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-dea8e16f3dd74291809c3038eb6c855c 2023-02-08 08:55:36,632 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-55af94394e4d4776a9a83012121fb840 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-55af94394e4d4776a9a83012121fb840 2023-02-08 08:55:36,633 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/b21a40c976d6492b80dffc0b17fd9c83_SeqId_34_ to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family1/b21a40c976d6492b80dffc0b17fd9c83_SeqId_34_ 2023-02-08 08:55:37,008 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/.tmp/family2/2261a79ec9b54b399457949b6df575db as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/2261a79ec9b54b399457949b6df575db 2023-02-08 08:55:37,019 INFO [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 5f0bc0fd5600fb2bfd54b18349c52a4a/family2 of 5f0bc0fd5600fb2bfd54b18349c52a4a into 2261a79ec9b54b399457949b6df575db(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:55:37,019 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5f0bc0fd5600fb2bfd54b18349c52a4a: 2023-02-08 08:55:37,019 INFO [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a., storeName=5f0bc0fd5600fb2bfd54b18349c52a4a/family2, priority=12, startTime=1675846534190; duration=0sec 2023-02-08 08:55:37,019 DEBUG [RS:0;jenkins-hbase4:42239-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:55:37,122 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-8eee8a31f2ca450389d1a3ee3f1350b0, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-dc241889a15f41ffbbbba991bb167763, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-eab5fe674d46495898c58cd9b8ca04bc, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-12f2df8055fc4e4da85427eb05d0bbff] to archive 2023-02-08 08:55:37,123 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:37,125 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-8eee8a31f2ca450389d1a3ee3f1350b0 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-8eee8a31f2ca450389d1a3ee3f1350b0 2023-02-08 08:55:37,126 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-dc241889a15f41ffbbbba991bb167763 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-dc241889a15f41ffbbbba991bb167763 2023-02-08 08:55:37,127 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-eab5fe674d46495898c58cd9b8ca04bc to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-eab5fe674d46495898c58cd9b8ca04bc 2023-02-08 08:55:37,129 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-12f2df8055fc4e4da85427eb05d0bbff to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/family2/testCompactionWithHFileLink=5f0bc0fd5600fb2bfd54b18349c52a4a-12f2df8055fc4e4da85427eb05d0bbff 2023-02-08 08:55:37,513 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:37,513 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:37,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-08 08:55:37,514 INFO [Listener at localhost/40763] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-02-08 08:55:37,519 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:37,520 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:37,520 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:37,522 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:37,522 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:37,522 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:37,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-08 08:55:37,524 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-08 08:55:37,524 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:37,524 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-08 08:55:37,524 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-08 08:55:37,525 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:37,525 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:37,526 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:37,526 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:37,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:37,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:37,526 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:37,526 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:37,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:37,527 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:37,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:37,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:37,527 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-08 08:55:37,527 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-08 08:55:37,527 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-08 08:55:37,527 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:37,527 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-08 08:55:37,527 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:55:37,527 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. started... 2023-02-08 08:55:37,527 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 08:55:37,528 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:37,529 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:55:37,528 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:55:37,529 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. started... 2023-02-08 08:55:37,529 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f19baf8f1950f69f65e90c2ac697eb02 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-08 08:55:37,529 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. started... 2023-02-08 08:55:37,530 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c4e6f28a0b52d9de77dc2ced57e02cc8 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-08 08:55:37,530 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 7e8631296641f454e22df0ec18084b7f 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-08 08:55:37,555 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302082a0bbf039bbd4aa597951a3799368546 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302082a0bbf039bbd4aa597951a3799368546 2023-02-08 08:55:37,957 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302083e2fc0983e654f528aa7e9727d14eaf0 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302083e2fc0983e654f528aa7e9727d14eaf0 2023-02-08 08:55:37,957 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302088b3e95238ab0414186dbe0dde364a36c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302088b3e95238ab0414186dbe0dde364a36c 2023-02-08 08:55:37,960 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family1/736348f7a8d14ac9a8b50b093f06fec6 2023-02-08 08:55:37,964 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family1/70a7bd35b35549cf9f2fb7eb8f41f221 2023-02-08 08:55:38,362 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family1/38d6f2072b0041349d0a3f57b5666ccc 2023-02-08 08:55:38,387 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302089edb32f4f1ab4ecda891a776fc573628 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202302089edb32f4f1ab4ecda891a776fc573628 2023-02-08 08:55:38,392 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208705936261ebc4862bc11f9e663591845 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230208705936261ebc4862bc11f9e663591845 2023-02-08 08:55:38,775 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302086dd21365baeb430f8f9b5f94926ff29a to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202302086dd21365baeb430f8f9b5f94926ff29a 2023-02-08 08:55:38,792 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family2/7d45372389c24a61b2d8705403e608b3 2023-02-08 08:55:38,795 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family1/736348f7a8d14ac9a8b50b093f06fec6 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/736348f7a8d14ac9a8b50b093f06fec6 2023-02-08 08:55:38,798 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/736348f7a8d14ac9a8b50b093f06fec6, entries=4, sequenceid=5, filesize=5.4 K 2023-02-08 08:55:38,799 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family2/7d45372389c24a61b2d8705403e608b3 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/7d45372389c24a61b2d8705403e608b3 2023-02-08 08:55:38,801 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/7d45372389c24a61b2d8705403e608b3, entries=2, sequenceid=5, filesize=5.1 K 2023-02-08 08:55:38,802 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 7e8631296641f454e22df0ec18084b7f in 1272ms, sequenceid=5, compaction requested=false 2023-02-08 08:55:38,802 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-02-08 08:55:38,803 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 7e8631296641f454e22df0ec18084b7f: 2023-02-08 08:55:38,803 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:55:38,803 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 08:55:38,805 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family2/1ee501dbda3d4f47833de2caa22d265d 2023-02-08 08:55:38,808 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family1/70a7bd35b35549cf9f2fb7eb8f41f221 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/70a7bd35b35549cf9f2fb7eb8f41f221 2023-02-08 08:55:38,811 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/70a7bd35b35549cf9f2fb7eb8f41f221, entries=4, sequenceid=5, filesize=5.4 K 2023-02-08 08:55:38,812 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family2/1ee501dbda3d4f47833de2caa22d265d as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/1ee501dbda3d4f47833de2caa22d265d 2023-02-08 08:55:38,815 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/1ee501dbda3d4f47833de2caa22d265d, entries=2, sequenceid=5, filesize=5.1 K 2023-02-08 08:55:38,816 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for f19baf8f1950f69f65e90c2ac697eb02 in 1287ms, sequenceid=5, compaction requested=false 2023-02-08 08:55:38,817 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f19baf8f1950f69f65e90c2ac697eb02: 2023-02-08 08:55:38,817 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:55:38,817 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 08:55:39,180 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family2/b9d4843276de4a1e97e2678412656d03 2023-02-08 08:55:39,184 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family1/38d6f2072b0041349d0a3f57b5666ccc as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/38d6f2072b0041349d0a3f57b5666ccc 2023-02-08 08:55:39,187 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/38d6f2072b0041349d0a3f57b5666ccc, entries=4, sequenceid=5, filesize=5.4 K 2023-02-08 08:55:39,187 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family2/b9d4843276de4a1e97e2678412656d03 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/b9d4843276de4a1e97e2678412656d03 2023-02-08 08:55:39,190 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/b9d4843276de4a1e97e2678412656d03, entries=2, sequenceid=5, filesize=5.1 K 2023-02-08 08:55:39,191 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for c4e6f28a0b52d9de77dc2ced57e02cc8 in 1661ms, sequenceid=5, compaction requested=false 2023-02-08 08:55:39,191 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c4e6f28a0b52d9de77dc2ced57e02cc8: 2023-02-08 08:55:39,191 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:55:39,191 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 08:55:39,192 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 08:55:39,192 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:39,192 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-08 08:55:39,192 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-08 08:55:39,193 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:39,193 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:39,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:39,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:39,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:39,194 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:39,194 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:39,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:39,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:39,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:39,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:39,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:39,196 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-08 08:55:39,196 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@71b9acff[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:39,196 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-08 08:55:39,196 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:39,197 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:39,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:39,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:39,197 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-08 08:55:39,197 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-08 08:55:39,197 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:55:39,197 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:39,197 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:39,198 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-08 08:55:39,198 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:39,198 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:39,198 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-08 08:55:39,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:39,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:39,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:39,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:39,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:39,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:39,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:39,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:39,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:39,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:39,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:55:39,201 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-08 08:55:39,201 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-08 08:55:39,201 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:39,201 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:39,201 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-08 08:55:39,201 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:39,204 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:39,204 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:39,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:39,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:39,204 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:39,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:39,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:39,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:39,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:39,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:39,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:39,205 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:39,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:39,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:39,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:39,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:39,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:39,207 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:39,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:39,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:39,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:39,209 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:39,209 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:39,209 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:39,209 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-08 08:55:39,209 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:39,209 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:39,209 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:39,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:39,209 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:39,210 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:39,210 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:39,210 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:39,210 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:39,210 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-08 08:55:39,210 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:39,210 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:39,210 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:39,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:39,310 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:39,310 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:39,310 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:39,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:39,312 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:39,312 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:39,312 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:39,314 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:39,314 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:39,315 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:39,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:39,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-08 08:55:39,317 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-08 08:55:39,317 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:39,317 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-08 08:55:39,317 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-08 08:55:39,317 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:39,317 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:39,318 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:39,318 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:39,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:39,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:39,318 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:39,318 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:39,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:39,319 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:39,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:39,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:39,319 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-08 08:55:39,319 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-08 08:55:39,319 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-08 08:55:39,320 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-08 08:55:39,320 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:39,320 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-08 08:55:39,320 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:55:39,320 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. started... 2023-02-08 08:55:39,320 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 08:55:39,320 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:39,320 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:55:39,320 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f19baf8f1950f69f65e90c2ac697eb02 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-08 08:55:39,320 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. started... 2023-02-08 08:55:39,320 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:55:39,320 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. started... 2023-02-08 08:55:39,320 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 7e8631296641f454e22df0ec18084b7f 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-08 08:55:39,320 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c4e6f28a0b52d9de77dc2ced57e02cc8 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-08 08:55:39,737 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208ea37caa6975d49dfbfedfb677a3d5f27 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230208ea37caa6975d49dfbfedfb677a3d5f27 2023-02-08 08:55:39,737 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208830fa4ce9bfb46fe841fccdfd918468f to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230208830fa4ce9bfb46fe841fccdfd918468f 2023-02-08 08:55:39,738 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208aaf9124bab294db6a3e043afd8263ef0 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230208aaf9124bab294db6a3e043afd8263ef0 2023-02-08 08:55:39,746 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family1/23f2fc2267e64787b7707d713d66eb78 2023-02-08 08:55:40,146 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family1/0bca660cc96445deb957998687f69e6a 2023-02-08 08:55:40,147 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family1/afce9bbeaa224dcc98c80d2d6c14e1e3 2023-02-08 08:55:40,165 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302082b5dbd84af194170b806ff26a75ecfec to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202302082b5dbd84af194170b806ff26a75ecfec 2023-02-08 08:55:40,173 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family2/85086f6b5afb41fea4381b8d358a34b5 2023-02-08 08:55:40,177 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family1/23f2fc2267e64787b7707d713d66eb78 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/23f2fc2267e64787b7707d713d66eb78 2023-02-08 08:55:40,180 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/23f2fc2267e64787b7707d713d66eb78, entries=4, sequenceid=9, filesize=5.4 K 2023-02-08 08:55:40,180 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family2/85086f6b5afb41fea4381b8d358a34b5 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/85086f6b5afb41fea4381b8d358a34b5 2023-02-08 08:55:40,183 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/85086f6b5afb41fea4381b8d358a34b5, entries=2, sequenceid=9, filesize=5.1 K 2023-02-08 08:55:40,184 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for c4e6f28a0b52d9de77dc2ced57e02cc8 in 864ms, sequenceid=9, compaction requested=false 2023-02-08 08:55:40,184 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c4e6f28a0b52d9de77dc2ced57e02cc8: 2023-02-08 08:55:40,184 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:55:40,184 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 08:55:40,567 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020849a4d9f127074a3b815c7ffd72553d1d to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c2023020849a4d9f127074a3b815c7ffd72553d1d 2023-02-08 08:55:40,567 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020802ded633207c418fa1fa377460968ea4 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023020802ded633207c418fa1fa377460968ea4 2023-02-08 08:55:40,571 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family2/4f2be3be079c4313ace709fb77f0d856 2023-02-08 08:55:40,575 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family1/afce9bbeaa224dcc98c80d2d6c14e1e3 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/afce9bbeaa224dcc98c80d2d6c14e1e3 2023-02-08 08:55:40,578 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/afce9bbeaa224dcc98c80d2d6c14e1e3, entries=4, sequenceid=9, filesize=5.4 K 2023-02-08 08:55:40,578 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family2/4f2be3be079c4313ace709fb77f0d856 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/4f2be3be079c4313ace709fb77f0d856 2023-02-08 08:55:40,581 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/4f2be3be079c4313ace709fb77f0d856, entries=2, sequenceid=9, filesize=5.1 K 2023-02-08 08:55:40,582 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for f19baf8f1950f69f65e90c2ac697eb02 in 1262ms, sequenceid=9, compaction requested=false 2023-02-08 08:55:40,582 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f19baf8f1950f69f65e90c2ac697eb02: 2023-02-08 08:55:40,582 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:55:40,582 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 08:55:40,972 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family2/4fc313bdbdd54297a8b938c078729229 2023-02-08 08:55:40,976 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family1/0bca660cc96445deb957998687f69e6a as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/0bca660cc96445deb957998687f69e6a 2023-02-08 08:55:40,979 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/0bca660cc96445deb957998687f69e6a, entries=4, sequenceid=9, filesize=5.4 K 2023-02-08 08:55:40,980 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family2/4fc313bdbdd54297a8b938c078729229 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/4fc313bdbdd54297a8b938c078729229 2023-02-08 08:55:40,983 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/4fc313bdbdd54297a8b938c078729229, entries=2, sequenceid=9, filesize=5.1 K 2023-02-08 08:55:40,983 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 7e8631296641f454e22df0ec18084b7f in 1663ms, sequenceid=9, compaction requested=false 2023-02-08 08:55:40,984 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 7e8631296641f454e22df0ec18084b7f: 2023-02-08 08:55:40,984 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:55:40,984 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 08:55:40,984 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 08:55:40,984 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:40,984 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-08 08:55:40,984 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-08 08:55:40,985 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:40,985 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:40,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:40,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:40,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:40,986 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:40,986 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:40,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:40,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:40,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:40,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:40,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:40,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-08 08:55:40,987 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@35beaef1[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:40,987 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-08 08:55:40,987 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:40,993 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:40,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:40,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:40,993 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-08 08:55:40,993 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-08 08:55:40,993 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:40,993 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:40,993 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:55:40,997 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:40,997 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-08 08:55:40,997 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:40,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:40,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:40,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:40,997 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-08 08:55:40,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:40,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:40,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:40,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:40,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:40,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:40,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:40,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:55:40,999 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-08 08:55:40,999 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-08 08:55:40,999 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:40,999 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:40,999 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-08 08:55:40,999 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:41,000 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:41,001 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:41,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:41,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:41,001 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:41,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:41,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:41,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:41,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:41,001 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:41,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:41,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:41,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:41,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:41,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:41,002 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:41,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:41,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:41,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:41,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:41,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:41,005 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:41,005 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:41,005 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:41,005 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:41,005 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:41,005 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:41,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-08 08:55:41,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:41,005 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:41,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:41,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:41,005 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:41,005 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-08 08:55:41,006 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-08 08:55:41,005 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:41,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:41,006 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:41,006 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:41,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:41,106 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:41,106 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:41,106 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:41,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:41,108 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:41,108 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:41,108 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:41,110 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:41,110 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:41,111 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:41,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:41,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-08 08:55:41,112 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-08 08:55:41,112 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:41,112 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-08 08:55:41,112 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-08 08:55:41,113 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:41,113 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:41,114 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:41,114 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:41,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:41,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:41,114 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:41,114 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:41,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:41,115 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:41,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:41,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:41,115 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-08 08:55:41,115 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-08 08:55:41,115 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-08 08:55:41,115 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-08 08:55:41,115 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:41,115 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-08 08:55:41,116 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:55:41,116 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. started... 2023-02-08 08:55:41,116 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 08:55:41,116 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:41,116 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:55:41,116 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f19baf8f1950f69f65e90c2ac697eb02 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-08 08:55:41,116 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. started... 2023-02-08 08:55:41,116 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:55:41,116 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. started... 2023-02-08 08:55:41,116 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 7e8631296641f454e22df0ec18084b7f 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-08 08:55:41,116 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c4e6f28a0b52d9de77dc2ced57e02cc8 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-08 08:55:41,130 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208a72eabe972254d8bbdc55b4e44ad7fc2 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230208a72eabe972254d8bbdc55b4e44ad7fc2 2023-02-08 08:55:41,530 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208b6d33a94c8f24d129769cd3494db5d37 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230208b6d33a94c8f24d129769cd3494db5d37 2023-02-08 08:55:41,530 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302083530f213599c417fbdf0a167a4b08c29 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302083530f213599c417fbdf0a167a4b08c29 2023-02-08 08:55:41,534 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family1/04dbb129a1574568b640afc685b40eff 2023-02-08 08:55:41,551 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208bd48e51f7b754a33822783ca969281cc to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230208bd48e51f7b754a33822783ca969281cc 2023-02-08 08:55:41,566 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family2/7b8013ba84014007976ed54ec0f49ace 2023-02-08 08:55:41,570 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family1/04dbb129a1574568b640afc685b40eff as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/04dbb129a1574568b640afc685b40eff 2023-02-08 08:55:41,574 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/04dbb129a1574568b640afc685b40eff, entries=4, sequenceid=13, filesize=5.4 K 2023-02-08 08:55:41,575 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family2/7b8013ba84014007976ed54ec0f49ace as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/7b8013ba84014007976ed54ec0f49ace 2023-02-08 08:55:41,578 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/7b8013ba84014007976ed54ec0f49ace, entries=2, sequenceid=13, filesize=5.1 K 2023-02-08 08:55:41,578 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 7e8631296641f454e22df0ec18084b7f in 462ms, sequenceid=13, compaction requested=true 2023-02-08 08:55:41,579 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 7e8631296641f454e22df0ec18084b7f: 2023-02-08 08:55:41,579 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:55:41,579 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 08:55:41,935 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family1/b7cd8310aa9a4f86ab172e4f1204313c 2023-02-08 08:55:41,935 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family1/51debd3e26cd4a57b0c6e75c3c8030f5 2023-02-08 08:55:42,048 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 08:55:42,348 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208360c7b1e937d40a4aed4041259672393 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230208360c7b1e937d40a4aed4041259672393 2023-02-08 08:55:42,348 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208b6a8c8dc733a476eaf2d462df802f0fa to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230208b6a8c8dc733a476eaf2d462df802f0fa 2023-02-08 08:55:42,753 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family2/d3ffc487031b471ea545dfe29f93f79f 2023-02-08 08:55:42,753 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family2/273c6f3fd6f041f6945c6d6cdb9ab64b 2023-02-08 08:55:42,757 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family1/b7cd8310aa9a4f86ab172e4f1204313c as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/b7cd8310aa9a4f86ab172e4f1204313c 2023-02-08 08:55:42,757 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family1/51debd3e26cd4a57b0c6e75c3c8030f5 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/51debd3e26cd4a57b0c6e75c3c8030f5 2023-02-08 08:55:42,760 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/b7cd8310aa9a4f86ab172e4f1204313c, entries=4, sequenceid=13, filesize=5.4 K 2023-02-08 08:55:42,760 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/51debd3e26cd4a57b0c6e75c3c8030f5, entries=4, sequenceid=13, filesize=5.4 K 2023-02-08 08:55:42,761 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family2/d3ffc487031b471ea545dfe29f93f79f as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/d3ffc487031b471ea545dfe29f93f79f 2023-02-08 08:55:42,761 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family2/273c6f3fd6f041f6945c6d6cdb9ab64b as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/273c6f3fd6f041f6945c6d6cdb9ab64b 2023-02-08 08:55:42,765 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/d3ffc487031b471ea545dfe29f93f79f, entries=2, sequenceid=13, filesize=5.1 K 2023-02-08 08:55:42,765 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/273c6f3fd6f041f6945c6d6cdb9ab64b, entries=2, sequenceid=13, filesize=5.1 K 2023-02-08 08:55:42,765 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for f19baf8f1950f69f65e90c2ac697eb02 in 1649ms, sequenceid=13, compaction requested=true 2023-02-08 08:55:42,766 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for c4e6f28a0b52d9de77dc2ced57e02cc8 in 1649ms, sequenceid=13, compaction requested=true 2023-02-08 08:55:42,766 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f19baf8f1950f69f65e90c2ac697eb02: 2023-02-08 08:55:42,766 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c4e6f28a0b52d9de77dc2ced57e02cc8: 2023-02-08 08:55:42,766 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:55:42,766 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:55:42,766 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 08:55:42,766 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 08:55:42,766 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 08:55:42,766 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:42,766 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-08 08:55:42,766 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-08 08:55:42,768 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:42,768 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:42,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:42,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:42,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:42,768 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:42,768 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:42,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:42,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:42,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:42,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:42,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:42,769 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-08 08:55:42,769 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2fb046fa[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:42,769 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-08 08:55:42,769 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:42,770 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:42,770 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:42,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:42,770 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-08 08:55:42,770 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-08 08:55:42,770 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:55:42,770 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:42,770 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:42,772 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:42,772 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-08 08:55:42,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:42,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:42,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:42,773 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:42,773 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-08 08:55:42,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:42,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:42,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:42,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:42,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:42,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:42,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:42,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:55:42,775 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-08 08:55:42,775 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-08 08:55:42,775 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:42,775 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:42,775 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-08 08:55:42,775 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:42,777 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:42,777 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:42,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:42,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:42,777 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:42,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:42,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:42,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:42,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:42,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:42,777 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:42,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:42,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:42,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:42,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:42,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:42,779 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:42,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:42,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:42,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:42,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:42,780 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:42,780 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:42,781 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:42,781 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-08 08:55:42,781 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:42,781 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:42,781 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-08 08:55:42,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:42,781 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:42,781 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-08 08:55:42,781 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:42,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:42,781 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:42,781 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:42,781 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:42,781 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:42,781 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:42,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:42,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:42,881 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:42,881 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:42,881 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:42,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:42,883 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:42,883 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:42,883 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:55:42,885 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:42,885 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:42,886 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:42,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:42,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-08 08:55:42,888 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-08 08:55:42,888 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:42,888 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-08 08:55:42,888 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-08 08:55:42,888 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:42,888 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:42,889 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:42,889 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:42,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:42,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:42,890 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:42,890 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:42,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:42,890 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:42,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:42,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:42,890 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-08 08:55:42,891 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-08 08:55:42,891 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-08 08:55:42,891 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-08 08:55:42,891 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:42,891 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-08 08:55:42,891 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:55:42,891 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. started... 2023-02-08 08:55:42,891 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:55:42,891 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 08:55:42,891 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f19baf8f1950f69f65e90c2ac697eb02 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-08 08:55:42,891 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:42,891 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:55:42,891 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. started... 2023-02-08 08:55:42,891 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. started... 2023-02-08 08:55:42,891 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c4e6f28a0b52d9de77dc2ced57e02cc8 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-08 08:55:42,891 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 7e8631296641f454e22df0ec18084b7f 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-08 08:55:42,906 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208251cbe6d83994a6cbea9732a0b391d87 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230208251cbe6d83994a6cbea9732a0b391d87 2023-02-08 08:55:43,308 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208e4d837984f5644a288171810f8ef5df1 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230208e4d837984f5644a288171810f8ef5df1 2023-02-08 08:55:43,309 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020885242b75e20442f988036b642733d94b to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023020885242b75e20442f988036b642733d94b 2023-02-08 08:55:43,311 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family1/b0b2666753ff407da6f3b3c7b4f26000 2023-02-08 08:55:43,341 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208ed152a910203423c976c0d72448f598c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230208ed152a910203423c976c0d72448f598c 2023-02-08 08:55:43,346 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family2/a5bd220a1f3f47c0bb9b598f48982836 2023-02-08 08:55:43,350 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family1/b0b2666753ff407da6f3b3c7b4f26000 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/b0b2666753ff407da6f3b3c7b4f26000 2023-02-08 08:55:43,354 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/b0b2666753ff407da6f3b3c7b4f26000, entries=4, sequenceid=17, filesize=5.4 K 2023-02-08 08:55:43,354 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family2/a5bd220a1f3f47c0bb9b598f48982836 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/a5bd220a1f3f47c0bb9b598f48982836 2023-02-08 08:55:43,358 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/a5bd220a1f3f47c0bb9b598f48982836, entries=2, sequenceid=17, filesize=5.1 K 2023-02-08 08:55:43,359 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 7e8631296641f454e22df0ec18084b7f in 468ms, sequenceid=17, compaction requested=true 2023-02-08 08:55:43,359 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 7e8631296641f454e22df0ec18084b7f: 2023-02-08 08:55:43,359 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:55:43,359 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 08:55:43,714 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family1/c8d89c873ecf44e0b94c06de2c2f064c 2023-02-08 08:55:43,724 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family1/73183e35e5e74ef883feaaf86b8b6cdf 2023-02-08 08:55:44,127 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020820976d01b4274034973860672bfed60f to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023020820976d01b4274034973860672bfed60f 2023-02-08 08:55:44,134 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020852327d4d4e744c34b734a490f8ad3db8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023020852327d4d4e744c34b734a490f8ad3db8 2023-02-08 08:55:44,531 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family2/619a8a985c7f4c1faaa7cd6e34b240ef 2023-02-08 08:55:44,535 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family1/c8d89c873ecf44e0b94c06de2c2f064c as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/c8d89c873ecf44e0b94c06de2c2f064c 2023-02-08 08:55:44,538 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family2/dd8b05d33eaf4684873bbaff576bce1f 2023-02-08 08:55:44,539 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/c8d89c873ecf44e0b94c06de2c2f064c, entries=4, sequenceid=17, filesize=5.4 K 2023-02-08 08:55:44,539 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family2/619a8a985c7f4c1faaa7cd6e34b240ef as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/619a8a985c7f4c1faaa7cd6e34b240ef 2023-02-08 08:55:44,542 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family1/73183e35e5e74ef883feaaf86b8b6cdf as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/73183e35e5e74ef883feaaf86b8b6cdf 2023-02-08 08:55:44,543 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/619a8a985c7f4c1faaa7cd6e34b240ef, entries=2, sequenceid=17, filesize=5.1 K 2023-02-08 08:55:44,544 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for f19baf8f1950f69f65e90c2ac697eb02 in 1653ms, sequenceid=17, compaction requested=true 2023-02-08 08:55:44,544 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f19baf8f1950f69f65e90c2ac697eb02: 2023-02-08 08:55:44,544 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:55:44,544 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 08:55:44,546 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/73183e35e5e74ef883feaaf86b8b6cdf, entries=4, sequenceid=17, filesize=5.4 K 2023-02-08 08:55:44,547 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family2/dd8b05d33eaf4684873bbaff576bce1f as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/dd8b05d33eaf4684873bbaff576bce1f 2023-02-08 08:55:44,551 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/dd8b05d33eaf4684873bbaff576bce1f, entries=2, sequenceid=17, filesize=5.1 K 2023-02-08 08:55:44,551 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for c4e6f28a0b52d9de77dc2ced57e02cc8 in 1660ms, sequenceid=17, compaction requested=true 2023-02-08 08:55:44,551 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c4e6f28a0b52d9de77dc2ced57e02cc8: 2023-02-08 08:55:44,551 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:55:44,552 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 08:55:44,552 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 08:55:44,552 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:44,552 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-08 08:55:44,552 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-08 08:55:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:44,554 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:44,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:44,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:44,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:44,554 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:44,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:44,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:44,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:44,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:44,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:44,556 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-08 08:55:44,556 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6cf56fed[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:44,556 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-08 08:55:44,556 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:44,557 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:44,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:44,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:44,557 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-08 08:55:44,557 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-08 08:55:44,557 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:44,557 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:44,557 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:55:44,558 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-08 08:55:44,558 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:44,558 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:44,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:44,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:44,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:44,558 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-08 08:55:44,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:44,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:44,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:44,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:44,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:44,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:44,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:44,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:55:44,561 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-08 08:55:44,561 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-08 08:55:44,561 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:44,561 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:44,561 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-08 08:55:44,561 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:44,562 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:44,562 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:44,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:44,562 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:44,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:44,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:44,563 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:44,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:44,564 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:44,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:44,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:44,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:44,566 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:44,566 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:44,566 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:44,566 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:44,566 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:44,566 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:44,566 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:44,566 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:44,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:44,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:44,566 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:44,566 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-08 08:55:44,567 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:44,566 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:44,567 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-08 08:55:44,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:44,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:44,567 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-08 08:55:44,567 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:44,667 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:44,667 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:44,667 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:44,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:44,680 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 08:55:44,690 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 08:55:44,701 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 08:55:44,702 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 08:55:44,729 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:44,729 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:44,730 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:44,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:44,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-08 08:55:44,731 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-08 08:55:44,731 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:44,731 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-08 08:55:44,731 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-08 08:55:44,732 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:44,732 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:44,733 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:44,733 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:44,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:44,733 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:44,733 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:44,734 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:44,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:44,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:44,734 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-08 08:55:44,734 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-08 08:55:44,734 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-08 08:55:44,734 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-08 08:55:44,734 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:44,734 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-08 08:55:44,734 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:55:44,734 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. started... 2023-02-08 08:55:44,734 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 08:55:44,734 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:55:44,735 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:55:44,735 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:44,734 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing f19baf8f1950f69f65e90c2ac697eb02 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-08 08:55:44,735 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. started... 2023-02-08 08:55:44,735 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. started... 2023-02-08 08:55:44,735 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c4e6f28a0b52d9de77dc2ced57e02cc8 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-08 08:55:44,735 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 7e8631296641f454e22df0ec18084b7f 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-08 08:55:45,556 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family1/d99ac18c925b4ecdbf2b44622b582de5 2023-02-08 08:55:45,556 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family1/a57a9fb678fd41cea183dbc6e6b4996a 2023-02-08 08:55:45,557 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family1/f0ab7e9abcfc4d98a6bfa6014eb2769b 2023-02-08 08:55:45,560 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a57a9fb678fd41cea183dbc6e6b4996a 2023-02-08 08:55:45,560 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d99ac18c925b4ecdbf2b44622b582de5 2023-02-08 08:55:45,560 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0ab7e9abcfc4d98a6bfa6014eb2769b 2023-02-08 08:55:45,975 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family2/97bb5ef970184e17a9e4059549a020f9 2023-02-08 08:55:45,980 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 97bb5ef970184e17a9e4059549a020f9 2023-02-08 08:55:45,985 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family1/a57a9fb678fd41cea183dbc6e6b4996a as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/a57a9fb678fd41cea183dbc6e6b4996a 2023-02-08 08:55:45,989 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a57a9fb678fd41cea183dbc6e6b4996a 2023-02-08 08:55:45,989 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/a57a9fb678fd41cea183dbc6e6b4996a, entries=3, sequenceid=23, filesize=5.0 K 2023-02-08 08:55:45,990 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family2/97bb5ef970184e17a9e4059549a020f9 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/97bb5ef970184e17a9e4059549a020f9 2023-02-08 08:55:45,994 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 97bb5ef970184e17a9e4059549a020f9 2023-02-08 08:55:45,994 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/97bb5ef970184e17a9e4059549a020f9, entries=1, sequenceid=23, filesize=5.0 K 2023-02-08 08:55:45,995 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 7e8631296641f454e22df0ec18084b7f in 1260ms, sequenceid=23, compaction requested=true 2023-02-08 08:55:45,995 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 7e8631296641f454e22df0ec18084b7f: 2023-02-08 08:55:45,995 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:55:45,995 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 08:55:46,382 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family2/cd09a58144d040d1aca4c5a3c56717f1 2023-02-08 08:55:46,382 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family2/ac33c85c1c0947da857a51b09536aa78 2023-02-08 08:55:46,385 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cd09a58144d040d1aca4c5a3c56717f1 2023-02-08 08:55:46,385 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ac33c85c1c0947da857a51b09536aa78 2023-02-08 08:55:46,386 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family1/d99ac18c925b4ecdbf2b44622b582de5 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/d99ac18c925b4ecdbf2b44622b582de5 2023-02-08 08:55:46,386 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family1/f0ab7e9abcfc4d98a6bfa6014eb2769b as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/f0ab7e9abcfc4d98a6bfa6014eb2769b 2023-02-08 08:55:46,389 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d99ac18c925b4ecdbf2b44622b582de5 2023-02-08 08:55:46,389 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/d99ac18c925b4ecdbf2b44622b582de5, entries=3, sequenceid=23, filesize=5.0 K 2023-02-08 08:55:46,390 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f0ab7e9abcfc4d98a6bfa6014eb2769b 2023-02-08 08:55:46,390 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/f0ab7e9abcfc4d98a6bfa6014eb2769b, entries=3, sequenceid=23, filesize=5.0 K 2023-02-08 08:55:46,390 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family2/cd09a58144d040d1aca4c5a3c56717f1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/cd09a58144d040d1aca4c5a3c56717f1 2023-02-08 08:55:46,391 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family2/ac33c85c1c0947da857a51b09536aa78 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/ac33c85c1c0947da857a51b09536aa78 2023-02-08 08:55:46,394 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cd09a58144d040d1aca4c5a3c56717f1 2023-02-08 08:55:46,394 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/cd09a58144d040d1aca4c5a3c56717f1, entries=1, sequenceid=23, filesize=5.0 K 2023-02-08 08:55:46,394 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ac33c85c1c0947da857a51b09536aa78 2023-02-08 08:55:46,395 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/ac33c85c1c0947da857a51b09536aa78, entries=1, sequenceid=23, filesize=5.0 K 2023-02-08 08:55:46,395 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for c4e6f28a0b52d9de77dc2ced57e02cc8 in 1660ms, sequenceid=23, compaction requested=true 2023-02-08 08:55:46,395 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c4e6f28a0b52d9de77dc2ced57e02cc8: 2023-02-08 08:55:46,395 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:55:46,395 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 08:55:46,395 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for f19baf8f1950f69f65e90c2ac697eb02 in 1661ms, sequenceid=23, compaction requested=true 2023-02-08 08:55:46,395 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for f19baf8f1950f69f65e90c2ac697eb02: 2023-02-08 08:55:46,395 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:55:46,395 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 08:55:46,395 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 08:55:46,395 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:46,396 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-08 08:55:46,396 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-08 08:55:46,397 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:46,398 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:46,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:46,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:46,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:46,398 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:46,398 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:46,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:46,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:46,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:46,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:46,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:46,399 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-08 08:55:46,400 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41df731f[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:46,400 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-08 08:55:46,400 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:46,401 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:46,401 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:46,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:46,401 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-08 08:55:46,401 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-08 08:55:46,401 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:55:46,401 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:46,401 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:46,403 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-08 08:55:46,403 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:46,403 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:46,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:46,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:46,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:46,403 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-08 08:55:46,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:46,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:46,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:46,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:46,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:46,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:46,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:46,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:55:46,406 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-08 08:55:46,406 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-08 08:55:46,406 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:46,406 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:46,406 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-08 08:55:46,406 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:46,410 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:46,410 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:46,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:46,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:46,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:46,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:46,410 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:46,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:46,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:46,411 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:46,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:46,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:46,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:46,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:46,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:46,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:46,412 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:46,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:46,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:46,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-08 08:55:46,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:46,414 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:46,414 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:46,414 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:46,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:46,415 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:46,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:46,418 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:46,418 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:46,418 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-08 08:55:46,418 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:46,418 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:46,418 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-08 08:55:46,418 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-08 08:55:46,418 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:46,418 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-08 08:55:46,418 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:46,419 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-08 08:55:46,419 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-08 08:55:46,419 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:46,519 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:46,519 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:46,519 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:46,520 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:46,520 DEBUG [Listener at localhost/40763] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 08:55:46,520 DEBUG [Listener at localhost/40763] regionserver.HStore(1912): f19baf8f1950f69f65e90c2ac697eb02/family1 is initiating major compaction (all files) 2023-02-08 08:55:46,520 INFO [Listener at localhost/40763] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:55:46,521 INFO [Listener at localhost/40763] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:55:46,521 INFO [Listener at localhost/40763] regionserver.HRegion(2258): Starting compaction of f19baf8f1950f69f65e90c2ac697eb02/family1 in ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:55:46,521 INFO [Listener at localhost/40763] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/70a7bd35b35549cf9f2fb7eb8f41f221, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/afce9bbeaa224dcc98c80d2d6c14e1e3, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/51debd3e26cd4a57b0c6e75c3c8030f5, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/c8d89c873ecf44e0b94c06de2c2f064c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/f0ab7e9abcfc4d98a6bfa6014eb2769b] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp, totalSize=26.6 K 2023-02-08 08:55:46,521 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 70a7bd35b35549cf9f2fb7eb8f41f221, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846537519 2023-02-08 08:55:46,522 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting afce9bbeaa224dcc98c80d2d6c14e1e3, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846539312 2023-02-08 08:55:46,522 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 51debd3e26cd4a57b0c6e75c3c8030f5, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846541108 2023-02-08 08:55:46,522 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting c8d89c873ecf44e0b94c06de2c2f064c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846542883 2023-02-08 08:55:46,523 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting f0ab7e9abcfc4d98a6bfa6014eb2769b, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 08:55:46,530 INFO [Listener at localhost/40763] throttle.PressureAwareThroughputController(145): f19baf8f1950f69f65e90c2ac697eb02#family1#compaction#149 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:46,937 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302083887a549f68d434aa59293fcb4b07baa_del 2023-02-08 08:55:46,938 INFO [Listener at localhost/40763] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302083887a549f68d434aa59293fcb4b07baa_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302083887a549f68d434aa59293fcb4b07baa_del 2023-02-08 08:55:47,750 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 27d66ff4744d40e8b40ee8fe2be0c723 2023-02-08 08:55:47,751 DEBUG [Listener at localhost/40763] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family1/27d66ff4744d40e8b40ee8fe2be0c723 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/27d66ff4744d40e8b40ee8fe2be0c723 2023-02-08 08:55:47,755 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 27d66ff4744d40e8b40ee8fe2be0c723 2023-02-08 08:55:47,756 INFO [Listener at localhost/40763] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in f19baf8f1950f69f65e90c2ac697eb02/family1 of f19baf8f1950f69f65e90c2ac697eb02 into 27d66ff4744d40e8b40ee8fe2be0c723(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 08:55:47,756 DEBUG [Listener at localhost/40763] regionserver.HRegion(2288): Compaction status journal for f19baf8f1950f69f65e90c2ac697eb02: 2023-02-08 08:55:47,756 DEBUG [Listener at localhost/40763] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 08:55:47,756 DEBUG [Listener at localhost/40763] regionserver.HStore(1912): f19baf8f1950f69f65e90c2ac697eb02/family2 is initiating major compaction (all files) 2023-02-08 08:55:47,756 INFO [Listener at localhost/40763] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:55:47,756 INFO [Listener at localhost/40763] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:55:47,756 INFO [Listener at localhost/40763] regionserver.HRegion(2258): Starting compaction of f19baf8f1950f69f65e90c2ac697eb02/family2 in ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:55:47,756 INFO [Listener at localhost/40763] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/1ee501dbda3d4f47833de2caa22d265d, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/4f2be3be079c4313ace709fb77f0d856, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/d3ffc487031b471ea545dfe29f93f79f, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/619a8a985c7f4c1faaa7cd6e34b240ef, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/ac33c85c1c0947da857a51b09536aa78] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp, totalSize=25.3 K 2023-02-08 08:55:47,757 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 1ee501dbda3d4f47833de2caa22d265d, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846537519 2023-02-08 08:55:47,757 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 4f2be3be079c4313ace709fb77f0d856, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846539312 2023-02-08 08:55:47,757 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting d3ffc487031b471ea545dfe29f93f79f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846541108 2023-02-08 08:55:47,758 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 619a8a985c7f4c1faaa7cd6e34b240ef, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846542883 2023-02-08 08:55:47,758 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting ac33c85c1c0947da857a51b09536aa78, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 08:55:47,768 INFO [Listener at localhost/40763] throttle.PressureAwareThroughputController(145): f19baf8f1950f69f65e90c2ac697eb02#family2#compaction#150 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:47,813 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/70a7bd35b35549cf9f2fb7eb8f41f221, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/afce9bbeaa224dcc98c80d2d6c14e1e3, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/51debd3e26cd4a57b0c6e75c3c8030f5, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/c8d89c873ecf44e0b94c06de2c2f064c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/f0ab7e9abcfc4d98a6bfa6014eb2769b] to archive 2023-02-08 08:55:47,813 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:47,815 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/70a7bd35b35549cf9f2fb7eb8f41f221 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/70a7bd35b35549cf9f2fb7eb8f41f221 2023-02-08 08:55:47,816 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/afce9bbeaa224dcc98c80d2d6c14e1e3 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/afce9bbeaa224dcc98c80d2d6c14e1e3 2023-02-08 08:55:47,817 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/51debd3e26cd4a57b0c6e75c3c8030f5 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/51debd3e26cd4a57b0c6e75c3c8030f5 2023-02-08 08:55:47,818 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/c8d89c873ecf44e0b94c06de2c2f064c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/c8d89c873ecf44e0b94c06de2c2f064c 2023-02-08 08:55:47,819 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/f0ab7e9abcfc4d98a6bfa6014eb2769b to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/f0ab7e9abcfc4d98a6bfa6014eb2769b 2023-02-08 08:55:48,175 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023020856adba7a9c8a43c493d6099d8b00d52f_del 2023-02-08 08:55:48,176 INFO [Listener at localhost/40763] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020856adba7a9c8a43c493d6099d8b00d52f_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e2023020856adba7a9c8a43c493d6099d8b00d52f_del 2023-02-08 08:55:48,988 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1645c7cf5234edd8cf1694d764a93de 2023-02-08 08:55:48,988 DEBUG [Listener at localhost/40763] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/.tmp/family2/c1645c7cf5234edd8cf1694d764a93de as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/c1645c7cf5234edd8cf1694d764a93de 2023-02-08 08:55:48,991 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c1645c7cf5234edd8cf1694d764a93de 2023-02-08 08:55:48,992 INFO [Listener at localhost/40763] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in f19baf8f1950f69f65e90c2ac697eb02/family2 of f19baf8f1950f69f65e90c2ac697eb02 into c1645c7cf5234edd8cf1694d764a93de(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 08:55:48,992 DEBUG [Listener at localhost/40763] regionserver.HRegion(2288): Compaction status journal for f19baf8f1950f69f65e90c2ac697eb02: 2023-02-08 08:55:48,993 DEBUG [Listener at localhost/40763] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 08:55:48,993 DEBUG [Listener at localhost/40763] regionserver.HStore(1912): 7e8631296641f454e22df0ec18084b7f/family1 is initiating major compaction (all files) 2023-02-08 08:55:48,993 INFO [Listener at localhost/40763] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:55:48,993 INFO [Listener at localhost/40763] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:55:48,994 INFO [Listener at localhost/40763] regionserver.HRegion(2258): Starting compaction of 7e8631296641f454e22df0ec18084b7f/family1 in ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:55:48,994 INFO [Listener at localhost/40763] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/736348f7a8d14ac9a8b50b093f06fec6, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/0bca660cc96445deb957998687f69e6a, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/04dbb129a1574568b640afc685b40eff, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/b0b2666753ff407da6f3b3c7b4f26000, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/a57a9fb678fd41cea183dbc6e6b4996a] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp, totalSize=26.6 K 2023-02-08 08:55:48,994 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 736348f7a8d14ac9a8b50b093f06fec6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846537520 2023-02-08 08:55:48,994 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 0bca660cc96445deb957998687f69e6a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846539312 2023-02-08 08:55:48,995 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 04dbb129a1574568b640afc685b40eff, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846541108 2023-02-08 08:55:48,995 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting b0b2666753ff407da6f3b3c7b4f26000, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846542883 2023-02-08 08:55:48,995 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting a57a9fb678fd41cea183dbc6e6b4996a, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 08:55:49,004 INFO [Listener at localhost/40763] throttle.PressureAwareThroughputController(145): 7e8631296641f454e22df0ec18084b7f#family1#compaction#151 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:49,011 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302088a6ee407e2924f658edf02fa6de668d0_del 2023-02-08 08:55:49,011 INFO [Listener at localhost/40763] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302088a6ee407e2924f658edf02fa6de668d0_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302088a6ee407e2924f658edf02fa6de668d0_del 2023-02-08 08:55:49,013 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/1ee501dbda3d4f47833de2caa22d265d, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/4f2be3be079c4313ace709fb77f0d856, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/d3ffc487031b471ea545dfe29f93f79f, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/619a8a985c7f4c1faaa7cd6e34b240ef, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/ac33c85c1c0947da857a51b09536aa78] to archive 2023-02-08 08:55:49,014 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:49,016 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/1ee501dbda3d4f47833de2caa22d265d to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/1ee501dbda3d4f47833de2caa22d265d 2023-02-08 08:55:49,017 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/4f2be3be079c4313ace709fb77f0d856 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/4f2be3be079c4313ace709fb77f0d856 2023-02-08 08:55:49,019 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/d3ffc487031b471ea545dfe29f93f79f to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/d3ffc487031b471ea545dfe29f93f79f 2023-02-08 08:55:49,020 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/619a8a985c7f4c1faaa7cd6e34b240ef to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/619a8a985c7f4c1faaa7cd6e34b240ef 2023-02-08 08:55:49,021 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/ac33c85c1c0947da857a51b09536aa78 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family2/ac33c85c1c0947da857a51b09536aa78 2023-02-08 08:55:49,823 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for adcd502aa78f423594a27d745cb635b7 2023-02-08 08:55:49,823 DEBUG [Listener at localhost/40763] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family1/adcd502aa78f423594a27d745cb635b7 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/adcd502aa78f423594a27d745cb635b7 2023-02-08 08:55:49,827 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for adcd502aa78f423594a27d745cb635b7 2023-02-08 08:55:49,828 INFO [Listener at localhost/40763] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 7e8631296641f454e22df0ec18084b7f/family1 of 7e8631296641f454e22df0ec18084b7f into adcd502aa78f423594a27d745cb635b7(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:55:49,828 DEBUG [Listener at localhost/40763] regionserver.HRegion(2288): Compaction status journal for 7e8631296641f454e22df0ec18084b7f: 2023-02-08 08:55:49,828 DEBUG [Listener at localhost/40763] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 08:55:49,828 DEBUG [Listener at localhost/40763] regionserver.HStore(1912): 7e8631296641f454e22df0ec18084b7f/family2 is initiating major compaction (all files) 2023-02-08 08:55:49,829 INFO [Listener at localhost/40763] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:55:49,829 INFO [Listener at localhost/40763] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:55:49,829 INFO [Listener at localhost/40763] regionserver.HRegion(2258): Starting compaction of 7e8631296641f454e22df0ec18084b7f/family2 in ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:55:49,829 INFO [Listener at localhost/40763] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/7d45372389c24a61b2d8705403e608b3, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/4fc313bdbdd54297a8b938c078729229, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/7b8013ba84014007976ed54ec0f49ace, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/a5bd220a1f3f47c0bb9b598f48982836, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/97bb5ef970184e17a9e4059549a020f9] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp, totalSize=25.3 K 2023-02-08 08:55:49,829 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 7d45372389c24a61b2d8705403e608b3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846537520 2023-02-08 08:55:49,830 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 4fc313bdbdd54297a8b938c078729229, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846539312 2023-02-08 08:55:49,830 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 7b8013ba84014007976ed54ec0f49ace, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846541108 2023-02-08 08:55:49,830 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting a5bd220a1f3f47c0bb9b598f48982836, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846542883 2023-02-08 08:55:49,831 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 97bb5ef970184e17a9e4059549a020f9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 08:55:49,838 INFO [Listener at localhost/40763] throttle.PressureAwareThroughputController(145): 7e8631296641f454e22df0ec18084b7f#family2#compaction#152 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:49,913 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/736348f7a8d14ac9a8b50b093f06fec6, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/0bca660cc96445deb957998687f69e6a, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/04dbb129a1574568b640afc685b40eff, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/b0b2666753ff407da6f3b3c7b4f26000, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/a57a9fb678fd41cea183dbc6e6b4996a] to archive 2023-02-08 08:55:49,913 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:49,915 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/736348f7a8d14ac9a8b50b093f06fec6 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/736348f7a8d14ac9a8b50b093f06fec6 2023-02-08 08:55:49,916 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/0bca660cc96445deb957998687f69e6a to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/0bca660cc96445deb957998687f69e6a 2023-02-08 08:55:49,917 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/04dbb129a1574568b640afc685b40eff to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/04dbb129a1574568b640afc685b40eff 2023-02-08 08:55:49,918 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/b0b2666753ff407da6f3b3c7b4f26000 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/b0b2666753ff407da6f3b3c7b4f26000 2023-02-08 08:55:49,919 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/a57a9fb678fd41cea183dbc6e6b4996a to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/a57a9fb678fd41cea183dbc6e6b4996a 2023-02-08 08:55:50,251 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302086443e6041fb64eb4aac20c83bebd71a4_del 2023-02-08 08:55:50,251 INFO [Listener at localhost/40763] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302086443e6041fb64eb4aac20c83bebd71a4_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202302086443e6041fb64eb4aac20c83bebd71a4_del 2023-02-08 08:55:50,276 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7bd7d9009b824bd5b9c1034b7babff5a 2023-02-08 08:55:50,277 DEBUG [Listener at localhost/40763] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/.tmp/family2/7bd7d9009b824bd5b9c1034b7babff5a as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/7bd7d9009b824bd5b9c1034b7babff5a 2023-02-08 08:55:50,282 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7bd7d9009b824bd5b9c1034b7babff5a 2023-02-08 08:55:50,283 INFO [Listener at localhost/40763] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 7e8631296641f454e22df0ec18084b7f/family2 of 7e8631296641f454e22df0ec18084b7f into 7bd7d9009b824bd5b9c1034b7babff5a(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:55:50,283 DEBUG [Listener at localhost/40763] regionserver.HRegion(2288): Compaction status journal for 7e8631296641f454e22df0ec18084b7f: 2023-02-08 08:55:50,283 DEBUG [Listener at localhost/40763] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 08:55:50,283 DEBUG [Listener at localhost/40763] regionserver.HStore(1912): c4e6f28a0b52d9de77dc2ced57e02cc8/family1 is initiating major compaction (all files) 2023-02-08 08:55:50,283 INFO [Listener at localhost/40763] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:55:50,283 INFO [Listener at localhost/40763] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:55:50,284 INFO [Listener at localhost/40763] regionserver.HRegion(2258): Starting compaction of c4e6f28a0b52d9de77dc2ced57e02cc8/family1 in ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:55:50,285 INFO [Listener at localhost/40763] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/38d6f2072b0041349d0a3f57b5666ccc, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/23f2fc2267e64787b7707d713d66eb78, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/b7cd8310aa9a4f86ab172e4f1204313c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/73183e35e5e74ef883feaaf86b8b6cdf, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/d99ac18c925b4ecdbf2b44622b582de5] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp, totalSize=26.6 K 2023-02-08 08:55:50,285 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 38d6f2072b0041349d0a3f57b5666ccc, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846537520 2023-02-08 08:55:50,286 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 23f2fc2267e64787b7707d713d66eb78, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846539312 2023-02-08 08:55:50,286 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting b7cd8310aa9a4f86ab172e4f1204313c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846541108 2023-02-08 08:55:50,286 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 73183e35e5e74ef883feaaf86b8b6cdf, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846542883 2023-02-08 08:55:50,287 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting d99ac18c925b4ecdbf2b44622b582de5, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 08:55:50,300 INFO [Listener at localhost/40763] throttle.PressureAwareThroughputController(145): c4e6f28a0b52d9de77dc2ced57e02cc8#family1#compaction#153 average throughput is 0.28 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:50,309 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230208522e4eaf51084c6ca13bdd340dc58577_del 2023-02-08 08:55:50,309 INFO [Listener at localhost/40763] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208522e4eaf51084c6ca13bdd340dc58577_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230208522e4eaf51084c6ca13bdd340dc58577_del 2023-02-08 08:55:50,315 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/7d45372389c24a61b2d8705403e608b3, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/4fc313bdbdd54297a8b938c078729229, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/7b8013ba84014007976ed54ec0f49ace, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/a5bd220a1f3f47c0bb9b598f48982836, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/97bb5ef970184e17a9e4059549a020f9] to archive 2023-02-08 08:55:50,316 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:50,319 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/7d45372389c24a61b2d8705403e608b3 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/7d45372389c24a61b2d8705403e608b3 2023-02-08 08:55:50,320 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/4fc313bdbdd54297a8b938c078729229 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/4fc313bdbdd54297a8b938c078729229 2023-02-08 08:55:50,322 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/7b8013ba84014007976ed54ec0f49ace to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/7b8013ba84014007976ed54ec0f49ace 2023-02-08 08:55:50,324 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/a5bd220a1f3f47c0bb9b598f48982836 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/a5bd220a1f3f47c0bb9b598f48982836 2023-02-08 08:55:50,325 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/97bb5ef970184e17a9e4059549a020f9 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family2/97bb5ef970184e17a9e4059549a020f9 2023-02-08 08:55:50,526 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 08:55:50,526 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x35721454 to 127.0.0.1:61594 2023-02-08 08:55:50,526 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:55:50,533 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 08:55:50,533 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5af1d7e5 to 127.0.0.1:61594 2023-02-08 08:55:50,533 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:55:51,124 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d1f597b58f514ef4a4d3611a9baf8cfc 2023-02-08 08:55:51,125 DEBUG [Listener at localhost/40763] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family1/d1f597b58f514ef4a4d3611a9baf8cfc as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/d1f597b58f514ef4a4d3611a9baf8cfc 2023-02-08 08:55:51,136 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d1f597b58f514ef4a4d3611a9baf8cfc 2023-02-08 08:55:51,138 INFO [Listener at localhost/40763] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in c4e6f28a0b52d9de77dc2ced57e02cc8/family1 of c4e6f28a0b52d9de77dc2ced57e02cc8 into d1f597b58f514ef4a4d3611a9baf8cfc(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:55:51,138 DEBUG [Listener at localhost/40763] regionserver.HRegion(2288): Compaction status journal for c4e6f28a0b52d9de77dc2ced57e02cc8: 2023-02-08 08:55:51,138 DEBUG [Listener at localhost/40763] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 08:55:51,138 DEBUG [Listener at localhost/40763] regionserver.HStore(1912): c4e6f28a0b52d9de77dc2ced57e02cc8/family2 is initiating major compaction (all files) 2023-02-08 08:55:51,138 INFO [Listener at localhost/40763] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:55:51,138 INFO [Listener at localhost/40763] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:55:51,139 INFO [Listener at localhost/40763] regionserver.HRegion(2258): Starting compaction of c4e6f28a0b52d9de77dc2ced57e02cc8/family2 in ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:55:51,139 INFO [Listener at localhost/40763] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/b9d4843276de4a1e97e2678412656d03, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/85086f6b5afb41fea4381b8d358a34b5, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/273c6f3fd6f041f6945c6d6cdb9ab64b, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/dd8b05d33eaf4684873bbaff576bce1f, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/cd09a58144d040d1aca4c5a3c56717f1] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp, totalSize=25.3 K 2023-02-08 08:55:51,140 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting b9d4843276de4a1e97e2678412656d03, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846537520 2023-02-08 08:55:51,140 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 85086f6b5afb41fea4381b8d358a34b5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846539312 2023-02-08 08:55:51,141 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 273c6f3fd6f041f6945c6d6cdb9ab64b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846541108 2023-02-08 08:55:51,141 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting dd8b05d33eaf4684873bbaff576bce1f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846542883 2023-02-08 08:55:51,141 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting cd09a58144d040d1aca4c5a3c56717f1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 08:55:51,150 INFO [Listener at localhost/40763] throttle.PressureAwareThroughputController(145): c4e6f28a0b52d9de77dc2ced57e02cc8#family2#compaction#154 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:55:51,218 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/38d6f2072b0041349d0a3f57b5666ccc, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/23f2fc2267e64787b7707d713d66eb78, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/b7cd8310aa9a4f86ab172e4f1204313c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/73183e35e5e74ef883feaaf86b8b6cdf, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/d99ac18c925b4ecdbf2b44622b582de5] to archive 2023-02-08 08:55:51,219 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:51,231 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/38d6f2072b0041349d0a3f57b5666ccc to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/38d6f2072b0041349d0a3f57b5666ccc 2023-02-08 08:55:51,233 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/23f2fc2267e64787b7707d713d66eb78 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/23f2fc2267e64787b7707d713d66eb78 2023-02-08 08:55:51,234 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/b7cd8310aa9a4f86ab172e4f1204313c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/b7cd8310aa9a4f86ab172e4f1204313c 2023-02-08 08:55:51,235 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/73183e35e5e74ef883feaaf86b8b6cdf to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/73183e35e5e74ef883feaaf86b8b6cdf 2023-02-08 08:55:51,237 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/d99ac18c925b4ecdbf2b44622b582de5 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/d99ac18c925b4ecdbf2b44622b582de5 2023-02-08 08:55:51,558 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230208824de959a23949b3b4b8f30d85d0e36e_del 2023-02-08 08:55:51,558 INFO [Listener at localhost/40763] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208824de959a23949b3b4b8f30d85d0e36e_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230208824de959a23949b3b4b8f30d85d0e36e_del 2023-02-08 08:55:52,371 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for be40b7da966140ba9942ffc90e83e05f 2023-02-08 08:55:52,371 DEBUG [Listener at localhost/40763] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/.tmp/family2/be40b7da966140ba9942ffc90e83e05f as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/be40b7da966140ba9942ffc90e83e05f 2023-02-08 08:55:52,375 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for be40b7da966140ba9942ffc90e83e05f 2023-02-08 08:55:52,376 INFO [Listener at localhost/40763] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in c4e6f28a0b52d9de77dc2ced57e02cc8/family2 of c4e6f28a0b52d9de77dc2ced57e02cc8 into be40b7da966140ba9942ffc90e83e05f(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 08:55:52,376 DEBUG [Listener at localhost/40763] regionserver.HRegion(2288): Compaction status journal for c4e6f28a0b52d9de77dc2ced57e02cc8: 2023-02-08 08:55:52,379 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 08:55:52,381 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 08:55:52,384 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 08:55:52,385 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 08:55:52,390 INFO [Listener at localhost/40763] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-08 08:55:52,403 INFO [Listener at localhost/40763] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-02-08 08:55:52,405 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302083887a549f68d434aa59293fcb4b07baa_del 2023-02-08 08:55:52,408 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230208522e4eaf51084c6ca13bdd340dc58577_del 2023-02-08 08:55:52,410 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302088a6ee407e2924f658edf02fa6de668d0_del 2023-02-08 08:55:52,410 INFO [Listener at localhost/40763] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-02-08 08:55:52,411 DEBUG [Listener at localhost/40763] zookeeper.ReadOnlyZKClient(139): Connect 0x4cb4d02d to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:55:52,414 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/b9d4843276de4a1e97e2678412656d03, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/85086f6b5afb41fea4381b8d358a34b5, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/273c6f3fd6f041f6945c6d6cdb9ab64b, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/dd8b05d33eaf4684873bbaff576bce1f, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/cd09a58144d040d1aca4c5a3c56717f1] to archive 2023-02-08 08:55:52,414 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:52,415 DEBUG [Listener at localhost/40763] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6e267277, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:55:52,416 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230208 for table='ns:testMinorCompaction' and column='family1' 2023-02-08 08:55:52,417 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230208 for table='ns:testMinorCompaction' and column='family1' 2023-02-08 08:55:52,416 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230208 for table='ns:testMinorCompaction' and column='family1' 2023-02-08 08:55:52,417 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/b9d4843276de4a1e97e2678412656d03 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/b9d4843276de4a1e97e2678412656d03 2023-02-08 08:55:52,418 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/85086f6b5afb41fea4381b8d358a34b5 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/85086f6b5afb41fea4381b8d358a34b5 2023-02-08 08:55:52,420 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/273c6f3fd6f041f6945c6d6cdb9ab64b to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/273c6f3fd6f041f6945c6d6cdb9ab64b 2023-02-08 08:55:52,422 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/dd8b05d33eaf4684873bbaff576bce1f to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/dd8b05d33eaf4684873bbaff576bce1f 2023-02-08 08:55:52,423 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/cd09a58144d040d1aca4c5a3c56717f1 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family2/cd09a58144d040d1aca4c5a3c56717f1 2023-02-08 08:55:52,448 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208298ae6fe188f4398876d4297ce6ba023 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230208298ae6fe188f4398876d4297ce6ba023 2023-02-08 08:55:52,449 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230208 table='ns:testMinorCompaction' column='family1' 2023-02-08 08:55:52,463 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:55:52,466 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60954, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:55:52,473 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:52,473 INFO [MobFileCompactionChore-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:52,474 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 08:55:52,476 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39684, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 08:55:52,486 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230208/family1/ffbac9ecf40a4e1b8fb4de0b5fc4cb5c first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-02-08 08:55:52,487 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230208/family1/ffbac9ecf40a4e1b8fb4de0b5fc4cb5c}] 2023-02-08 08:55:52,496 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230208/family1/ffbac9ecf40a4e1b8fb4de0b5fc4cb5c for inclusion in f19baf8f1950f69f65e90c2ac697eb02/family1 2023-02-08 08:55:52,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-02-08 08:55:52,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last=1 2023-02-08 08:55:52,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for f19baf8f1950f69f65e90c2ac697eb02: 2023-02-08 08:55:52,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230208/family1/ffbac9ecf40a4e1b8fb4de0b5fc4cb5c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__ns_testMinorCompaction__7pkl080oj6gqpfcmka4tah8l1nr7old29mbu7r6kses6un7cvcs5fim27ssost9r/family1/ffbac9ecf40a4e1b8fb4de0b5fc4cb5c 2023-02-08 08:55:52,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__ns_testMinorCompaction__7pkl080oj6gqpfcmka4tah8l1nr7old29mbu7r6kses6un7cvcs5fim27ssost9r/family1/ffbac9ecf40a4e1b8fb4de0b5fc4cb5c as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/5c0dce5f73704642860c30c6b27d5054_SeqId_28_ 2023-02-08 08:55:52,501 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__ns_testMinorCompaction__7pkl080oj6gqpfcmka4tah8l1nr7old29mbu7r6kses6un7cvcs5fim27ssost9r/family1/ffbac9ecf40a4e1b8fb4de0b5fc4cb5c into f19baf8f1950f69f65e90c2ac697eb02/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/5c0dce5f73704642860c30c6b27d5054_SeqId_28_ - updating store file list. 2023-02-08 08:55:52,504 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/5c0dce5f73704642860c30c6b27d5054_SeqId_28_ into f19baf8f1950f69f65e90c2ac697eb02/family1 2023-02-08 08:55:52,504 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__ns_testMinorCompaction__7pkl080oj6gqpfcmka4tah8l1nr7old29mbu7r6kses6un7cvcs5fim27ssost9r/family1/ffbac9ecf40a4e1b8fb4de0b5fc4cb5c into f19baf8f1950f69f65e90c2ac697eb02/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/family1/5c0dce5f73704642860c30c6b27d5054_SeqId_28_) 2023-02-08 08:55:52,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__ns_testMinorCompaction__7pkl080oj6gqpfcmka4tah8l1nr7old29mbu7r6kses6un7cvcs5fim27ssost9r/family1/ffbac9ecf40a4e1b8fb4de0b5fc4cb5c 2023-02-08 08:55:52,506 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230208 table='ns:testMinorCompaction' column='family1' 2023-02-08 08:55:52,507 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:52,508 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302088b3e95238ab0414186dbe0dde364a36c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302088b3e95238ab0414186dbe0dde364a36c 2023-02-08 08:55:52,508 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-08 08:55:52,508 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230208' 2023-02-08 08:55:52,509 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230208 for table='ns:testMinorCompaction' and column='family1' 2023-02-08 08:55:53,246 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302088db3f4a4c66a468fb9ec38198c01dde4 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302088db3f4a4c66a468fb9ec38198c01dde4 2023-02-08 08:55:53,247 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230208 table='ns:testMinorCompaction' column='family1' 2023-02-08 08:55:53,248 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020846994afcf89b4a6f9d3b0dd99b896c03 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023020846994afcf89b4a6f9d3b0dd99b896c03 2023-02-08 08:55:53,249 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230208 table='ns:testMinorCompaction' column='family1' 2023-02-08 08:55:53,262 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:53,262 INFO [MobFileCompactionChore-pool-2] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:53,263 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:53,263 INFO [MobFileCompactionChore-pool-1] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:53,272 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230208/family1/96a691b5b2574bfdb9c596013f325b3b first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-02-08 08:55:53,273 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230208/family1/96a691b5b2574bfdb9c596013f325b3b}] 2023-02-08 08:55:53,273 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230208/family1/4e2e1d4c0de34d3fb85d864997fd45bf first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-02-08 08:55:53,273 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230208/family1/4e2e1d4c0de34d3fb85d864997fd45bf}] 2023-02-08 08:55:53,275 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230208/family1/96a691b5b2574bfdb9c596013f325b3b for inclusion in c4e6f28a0b52d9de77dc2ced57e02cc8/family1 2023-02-08 08:55:53,275 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230208/family1/4e2e1d4c0de34d3fb85d864997fd45bf for inclusion in 7e8631296641f454e22df0ec18084b7f/family1 2023-02-08 08:55:53,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-02-08 08:55:53,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-08 08:55:53,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-02-08 08:55:53,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-08 08:55:53,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for c4e6f28a0b52d9de77dc2ced57e02cc8: 2023-02-08 08:55:53,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for 7e8631296641f454e22df0ec18084b7f: 2023-02-08 08:55:53,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230208/family1/96a691b5b2574bfdb9c596013f325b3b to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__ns_testMinorCompaction__mukncvqpnf0oht13jadbmtau7fjae5bn7k8sknomrtgia872lqlck3ivgihn0de/family1/96a691b5b2574bfdb9c596013f325b3b 2023-02-08 08:55:53,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230208/family1/4e2e1d4c0de34d3fb85d864997fd45bf to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__ns_testMinorCompaction__c16aqpfm08teqrnqj0mqe8iijtu2o20vq1v75v4b4lguab0bb70njmrc1225p87q/family1/4e2e1d4c0de34d3fb85d864997fd45bf 2023-02-08 08:55:53,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__ns_testMinorCompaction__mukncvqpnf0oht13jadbmtau7fjae5bn7k8sknomrtgia872lqlck3ivgihn0de/family1/96a691b5b2574bfdb9c596013f325b3b as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/da1136ae8c9747089e4d5fb9230801c4_SeqId_28_ 2023-02-08 08:55:53,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__ns_testMinorCompaction__c16aqpfm08teqrnqj0mqe8iijtu2o20vq1v75v4b4lguab0bb70njmrc1225p87q/family1/4e2e1d4c0de34d3fb85d864997fd45bf as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/34506aef9fbb44be9cce913f6c54c1f6_SeqId_28_ 2023-02-08 08:55:53,280 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__ns_testMinorCompaction__mukncvqpnf0oht13jadbmtau7fjae5bn7k8sknomrtgia872lqlck3ivgihn0de/family1/96a691b5b2574bfdb9c596013f325b3b into c4e6f28a0b52d9de77dc2ced57e02cc8/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/da1136ae8c9747089e4d5fb9230801c4_SeqId_28_ - updating store file list. 2023-02-08 08:55:53,280 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__ns_testMinorCompaction__c16aqpfm08teqrnqj0mqe8iijtu2o20vq1v75v4b4lguab0bb70njmrc1225p87q/family1/4e2e1d4c0de34d3fb85d864997fd45bf into 7e8631296641f454e22df0ec18084b7f/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/34506aef9fbb44be9cce913f6c54c1f6_SeqId_28_ - updating store file list. 2023-02-08 08:55:53,283 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/da1136ae8c9747089e4d5fb9230801c4_SeqId_28_ into c4e6f28a0b52d9de77dc2ced57e02cc8/family1 2023-02-08 08:55:53,283 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/34506aef9fbb44be9cce913f6c54c1f6_SeqId_28_ into 7e8631296641f454e22df0ec18084b7f/family1 2023-02-08 08:55:53,283 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__ns_testMinorCompaction__mukncvqpnf0oht13jadbmtau7fjae5bn7k8sknomrtgia872lqlck3ivgihn0de/family1/96a691b5b2574bfdb9c596013f325b3b into c4e6f28a0b52d9de77dc2ced57e02cc8/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/family1/da1136ae8c9747089e4d5fb9230801c4_SeqId_28_) 2023-02-08 08:55:53,283 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__ns_testMinorCompaction__c16aqpfm08teqrnqj0mqe8iijtu2o20vq1v75v4b4lguab0bb70njmrc1225p87q/family1/4e2e1d4c0de34d3fb85d864997fd45bf into 7e8631296641f454e22df0ec18084b7f/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/family1/34506aef9fbb44be9cce913f6c54c1f6_SeqId_28_) 2023-02-08 08:55:53,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__ns_testMinorCompaction__mukncvqpnf0oht13jadbmtau7fjae5bn7k8sknomrtgia872lqlck3ivgihn0de/family1/96a691b5b2574bfdb9c596013f325b3b 2023-02-08 08:55:53,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__ns_testMinorCompaction__c16aqpfm08teqrnqj0mqe8iijtu2o20vq1v75v4b4lguab0bb70njmrc1225p87q/family1/4e2e1d4c0de34d3fb85d864997fd45bf 2023-02-08 08:55:53,285 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230208 table='ns:testMinorCompaction' column='family1' 2023-02-08 08:55:53,285 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230208 table='ns:testMinorCompaction' column='family1' 2023-02-08 08:55:53,285 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:53,286 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:55:53,287 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302083e2fc0983e654f528aa7e9727d14eaf0 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302083e2fc0983e654f528aa7e9727d14eaf0 2023-02-08 08:55:53,287 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-08 08:55:53,287 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302082a0bbf039bbd4aa597951a3799368546 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302082a0bbf039bbd4aa597951a3799368546 2023-02-08 08:55:53,287 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230208' 2023-02-08 08:55:53,287 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-08 08:55:53,287 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230208' 2023-02-08 08:55:53,287 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230208 for table='ns:testMinorCompaction' and column='family1' 2023-02-08 08:55:53,287 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230208 for table='ns:testMinorCompaction' and column='family1' 2023-02-08 08:55:53,287 INFO [Listener at localhost/40763] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-02-08 08:55:53,289 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 08:55:53,291 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 08:55:53,293 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 08:55:53,294 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 08:55:53,309 INFO [Listener at localhost/40763] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=248 (was 285), OpenFileDescriptor=621 (was 767), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=120 (was 136), ProcessCount=168 (was 168), AvailableMemoryMB=3914 (was 4041) 2023-02-08 08:55:53,323 INFO [Listener at localhost/40763] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=248, OpenFileDescriptor=621, MaxFileDescriptor=60000, SystemLoadAverage=120, ProcessCount=168, AvailableMemoryMB=4179 2023-02-08 08:55:53,324 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:53,324 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:53,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-08 08:55:53,326 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:53,328 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:55:53,328 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-02-08 08:55:53,328 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:53,328 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:53,328 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:55:53,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-08 08:55:53,330 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121 2023-02-08 08:55:53,330 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121 empty. 2023-02-08 08:55:53,331 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121 2023-02-08 08:55:53,331 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-02-08 08:55:53,429 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:53,429 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:53,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-08 08:55:53,630 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:53,631 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:53,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-08 08:55:53,738 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-08 08:55:53,739 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 70af1f4bea4eb4fc084512e4e5840121, NAME => 'testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp 2023-02-08 08:55:53,932 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:53,932 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:53,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-08 08:55:54,144 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:54,144 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1603): Closing 70af1f4bea4eb4fc084512e4e5840121, disabling compactions & flushes 2023-02-08 08:55:54,145 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:55:54,145 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:55:54,145 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. after waiting 0 ms 2023-02-08 08:55:54,145 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:55:54,145 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1837): Closed testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:55:54,145 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 70af1f4bea4eb4fc084512e4e5840121: Waiting for close lock at 1675846554144Disabling compacts and flushes for region at 1675846554144Disabling writes for close at 1675846554145 (+1 ms)Writing region close event to WAL at 1675846554145Closed at 1675846554145 2023-02-08 08:55:54,147 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:55:54,148 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675846554148"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846554148"}]},"ts":"1675846554148"} 2023-02-08 08:55:54,149 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-08 08:55:54,150 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:55:54,150 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846554150"}]},"ts":"1675846554150"} 2023-02-08 08:55:54,151 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-02-08 08:55:54,158 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=70af1f4bea4eb4fc084512e4e5840121, ASSIGN}] 2023-02-08 08:55:54,160 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=70af1f4bea4eb4fc084512e4e5840121, ASSIGN 2023-02-08 08:55:54,161 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=70af1f4bea4eb4fc084512e4e5840121, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=false 2023-02-08 08:55:54,312 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=70af1f4bea4eb4fc084512e4e5840121, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:54,312 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675846554312"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846554312"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846554312"}]},"ts":"1675846554312"} 2023-02-08 08:55:54,313 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure 70af1f4bea4eb4fc084512e4e5840121, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:55:54,433 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:54,434 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:54,435 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-08 08:55:54,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:55:54,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 70af1f4bea4eb4fc084512e4e5840121, NAME => 'testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121.', STARTKEY => '', ENDKEY => ''} 2023-02-08 08:55:54,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy 70af1f4bea4eb4fc084512e4e5840121 2023-02-08 08:55:54,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:55:54,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 70af1f4bea4eb4fc084512e4e5840121 2023-02-08 08:55:54,468 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 70af1f4bea4eb4fc084512e4e5840121 2023-02-08 08:55:54,469 INFO [StoreOpener-70af1f4bea4eb4fc084512e4e5840121-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 70af1f4bea4eb4fc084512e4e5840121 2023-02-08 08:55:54,471 DEBUG [StoreOpener-70af1f4bea4eb4fc084512e4e5840121-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1 2023-02-08 08:55:54,471 DEBUG [StoreOpener-70af1f4bea4eb4fc084512e4e5840121-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1 2023-02-08 08:55:54,471 INFO [StoreOpener-70af1f4bea4eb4fc084512e4e5840121-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 70af1f4bea4eb4fc084512e4e5840121 columnFamilyName family1 2023-02-08 08:55:54,473 INFO [StoreOpener-70af1f4bea4eb4fc084512e4e5840121-1] regionserver.HStore(310): Store=70af1f4bea4eb4fc084512e4e5840121/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:55:54,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121 2023-02-08 08:55:54,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121 2023-02-08 08:55:54,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 70af1f4bea4eb4fc084512e4e5840121 2023-02-08 08:55:54,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:55:54,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 70af1f4bea4eb4fc084512e4e5840121; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75301084, jitterRate=0.12207359075546265}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-08 08:55:54,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 70af1f4bea4eb4fc084512e4e5840121: Running coprocessor pre-open hook at 1675846554468Writing region info on filesystem at 1675846554468Initializing all the Stores at 1675846554469 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1675846554469Cleaning up temporary data from old regions at 1675846554475 (+6 ms)Cleaning up detritus from prior splits at 1675846554475Running coprocessor post-open hooks at 1675846554479 (+4 ms)Region opened successfully at 1675846554479 2023-02-08 08:55:54,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121., pid=114, masterSystemTime=1675846554465 2023-02-08 08:55:54,482 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:55:54,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:55:54,482 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=70af1f4bea4eb4fc084512e4e5840121, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:54,482 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1675846554482"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846554482"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846554482"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846554482"}]},"ts":"1675846554482"} 2023-02-08 08:55:54,485 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-08 08:55:54,485 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure 70af1f4bea4eb4fc084512e4e5840121, server=jenkins-hbase4.apache.org,42239,1675846413348 in 171 msec 2023-02-08 08:55:54,486 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-08 08:55:54,487 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=70af1f4bea4eb4fc084512e4e5840121, ASSIGN in 327 msec 2023-02-08 08:55:54,487 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:55:54,487 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846554487"}]},"ts":"1675846554487"} 2023-02-08 08:55:54,488 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-02-08 08:55:54,490 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:55:54,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 1.1640 sec 2023-02-08 08:55:55,435 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:55,435 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:55,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-08 08:55:55,436 INFO [Listener at localhost/40763] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-02-08 08:55:55,458 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:55,458 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:55,458 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:55,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,464 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 08:55:55,464 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:55,464 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 08:55:55,464 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 08:55:55,464 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,464 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,466 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:55,466 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:55,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:55,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:55,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,466 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:55,466 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:55,466 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:55,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,467 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,467 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,467 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 08:55:55,467 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:55,467 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 08:55:55,469 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:55,469 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:55,469 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:55:55,469 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. started... 2023-02-08 08:55:55,470 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 70af1f4bea4eb4fc084512e4e5840121 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-08 08:55:55,494 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130a39ecbb027f6437eb9d40930132197eb to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130a39ecbb027f6437eb9d40930132197eb 2023-02-08 08:55:55,897 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/2b8ae81853414d7797c207b766994168 2023-02-08 08:55:55,902 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/2b8ae81853414d7797c207b766994168 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/2b8ae81853414d7797c207b766994168 2023-02-08 08:55:55,905 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/2b8ae81853414d7797c207b766994168, entries=1000, sequenceid=5, filesize=166.0 K 2023-02-08 08:55:55,906 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 70af1f4bea4eb4fc084512e4e5840121 in 436ms, sequenceid=5, compaction requested=false 2023-02-08 08:55:55,906 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 08:55:55,907 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 70af1f4bea4eb4fc084512e4e5840121: 2023-02-08 08:55:55,907 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:55:55,907 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:55,907 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:55,907 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:55,907 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 08:55:55,907 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-08 08:55:55,909 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,909 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:55,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:55,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:55,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:55,910 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,910 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:55,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:55,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:55,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:55,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:55,911 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 08:55:55,911 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@21c9784c[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:55,911 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 08:55:55,911 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,912 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,912 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,912 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 08:55:55,912 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-08 08:55:55,912 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:55:55,912 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:55,912 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:55,913 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 08:55:55,913 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:55,913 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:55,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:55,914 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-08 08:55:55,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:55,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:55,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:55,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:55,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:55,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:55,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:55,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:55:55,916 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:55:55,916 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-08 08:55:55,916 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:55,916 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:55,916 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,916 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:55,918 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,918 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:55,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:55,918 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:55,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:55,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:55,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:55,919 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:55,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:55,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,920 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:55,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:55,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:55,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:55,923 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:55,923 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,923 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,923 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:55,923 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,923 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 08:55:55,923 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,923 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:55,924 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:55,924 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:55,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:55,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:55,924 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:55,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:55,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:55,932 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 08:55:55,932 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:55,932 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:55:55,932 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:56,032 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:56,033 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:56,033 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:56,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:56,046 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:56,046 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:56,047 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:56,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:56,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,052 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 08:55:56,052 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:56,052 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 08:55:56,052 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 08:55:56,053 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,053 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,054 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:56,054 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:56,054 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:56,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:56,054 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:56,054 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:56,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,055 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:56,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,055 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,055 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 08:55:56,055 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,055 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 08:55:56,055 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:56,055 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 08:55:56,056 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:56,056 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:56,056 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:55:56,056 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. started... 2023-02-08 08:55:56,056 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 70af1f4bea4eb4fc084512e4e5840121 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-08 08:55:56,078 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511283fd2061d56af4272a57c48ace01928e6 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511283fd2061d56af4272a57c48ace01928e6 2023-02-08 08:55:56,086 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/05135a0a28bf4168b73d32fb60984230 2023-02-08 08:55:56,091 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/05135a0a28bf4168b73d32fb60984230 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/05135a0a28bf4168b73d32fb60984230 2023-02-08 08:55:56,095 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/05135a0a28bf4168b73d32fb60984230, entries=1, sequenceid=9, filesize=5.0 K 2023-02-08 08:55:56,096 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 70af1f4bea4eb4fc084512e4e5840121 in 40ms, sequenceid=9, compaction requested=false 2023-02-08 08:55:56,096 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 70af1f4bea4eb4fc084512e4e5840121: 2023-02-08 08:55:56,096 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:55:56,096 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:56,096 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:56,096 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:56,096 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 08:55:56,097 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-08 08:55:56,098 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,098 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:56,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:56,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:56,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:56,099 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,099 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:56,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:56,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:56,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:56,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:56,100 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 08:55:56,100 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69d0806d[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:56,100 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 08:55:56,100 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,102 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,102 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 08:55:56,102 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-08 08:55:56,102 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:55:56,102 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:56,102 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:56,104 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:56,104 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 08:55:56,104 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:56,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:56,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:56,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:56,104 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-08 08:55:56,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:56,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:56,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:56,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:56,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:56,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:55:56,107 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:55:56,107 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-08 08:55:56,107 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:56,107 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:56,107 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,107 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:56,109 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,109 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:56,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,109 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:56,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:56,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:56,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:56,109 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:56,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:56,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:56,110 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:56,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:56,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:56,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:56,113 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:56,113 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,113 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:56,113 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:56,113 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 08:55:56,113 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:56,113 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:56,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:56,113 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:55:56,114 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:56,113 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:56,113 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 08:55:56,113 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:56,114 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,114 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,114 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,114 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:56,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:56,214 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:56,214 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:56,214 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:56,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:56,219 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:56,219 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:56,219 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:56,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:56,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,221 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 08:55:56,221 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:56,221 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 08:55:56,221 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 08:55:56,222 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,222 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,228 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:56,228 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:56,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:56,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:56,229 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:56,229 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:56,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,229 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:56,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,230 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,230 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 08:55:56,230 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:56,230 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 08:55:56,230 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:56,230 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 08:55:56,230 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:56,230 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:56,230 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:55:56,230 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. started... 2023-02-08 08:55:56,230 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 70af1f4bea4eb4fc084512e4e5840121 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:55:56,654 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512011aaf9f12132548cdaf8a35903df2baa6 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512011aaf9f12132548cdaf8a35903df2baa6 2023-02-08 08:55:57,059 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/4e4cfc6cfc7b458f8cd951605f739daf 2023-02-08 08:55:57,063 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/4e4cfc6cfc7b458f8cd951605f739daf as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/4e4cfc6cfc7b458f8cd951605f739daf 2023-02-08 08:55:57,067 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/4e4cfc6cfc7b458f8cd951605f739daf, entries=1, sequenceid=13, filesize=4.9 K 2023-02-08 08:55:57,068 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 70af1f4bea4eb4fc084512e4e5840121 in 838ms, sequenceid=13, compaction requested=true 2023-02-08 08:55:57,068 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 70af1f4bea4eb4fc084512e4e5840121: 2023-02-08 08:55:57,068 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:55:57,068 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:57,068 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:57,068 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,068 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 08:55:57,068 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-08 08:55:57,070 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,070 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:57,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:57,070 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,070 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:57,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:57,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:57,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:57,072 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 08:55:57,072 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2e9d10bd[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:57,072 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 08:55:57,072 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,073 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,073 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 08:55:57,073 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-08 08:55:57,074 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:55:57,074 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,074 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:57,075 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,075 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 08:55:57,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:57,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:57,075 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:57,075 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-08 08:55:57,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:57,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:57,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:57,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:55:57,077 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:55:57,077 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-08 08:55:57,077 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:57,077 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:57,077 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,077 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:57,080 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,080 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,080 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:57,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:57,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:57,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:57,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:57,080 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:57,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:57,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,081 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:57,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,083 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:57,084 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:57,083 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:57,084 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 08:55:57,084 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:57,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 08:55:57,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:57,084 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:57,084 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,084 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,084 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,084 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,084 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,084 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:55:57,084 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:57,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:57,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:57,185 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:57,185 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:57,185 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:57,186 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:57,189 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:57,189 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:57,190 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:57,191 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:57,192 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,192 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 08:55:57,192 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:57,192 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 08:55:57,192 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 08:55:57,192 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,192 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,194 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:57,194 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:57,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:57,194 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,194 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:57,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:57,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,195 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,195 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 08:55:57,195 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,195 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 08:55:57,195 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:57,195 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 08:55:57,195 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:57,195 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:57,195 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:55:57,196 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. started... 2023-02-08 08:55:57,196 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 70af1f4bea4eb4fc084512e4e5840121 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:55:57,206 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205ab29dcc763db4adab7abddfcc27ea3b6 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205ab29dcc763db4adab7abddfcc27ea3b6 2023-02-08 08:55:57,210 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/7e94dc57529c41fea673a8e7604137cd 2023-02-08 08:55:57,214 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/7e94dc57529c41fea673a8e7604137cd as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/7e94dc57529c41fea673a8e7604137cd 2023-02-08 08:55:57,217 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/7e94dc57529c41fea673a8e7604137cd, entries=1, sequenceid=17, filesize=4.9 K 2023-02-08 08:55:57,218 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 70af1f4bea4eb4fc084512e4e5840121 in 22ms, sequenceid=17, compaction requested=true 2023-02-08 08:55:57,218 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 70af1f4bea4eb4fc084512e4e5840121: 2023-02-08 08:55:57,218 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:55:57,218 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:57,218 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:57,218 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,219 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 08:55:57,219 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-08 08:55:57,220 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,220 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:57,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:57,220 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,220 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:57,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:57,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:57,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:57,222 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 08:55:57,222 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@14e0d9fa[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:57,222 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 08:55:57,222 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,226 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,227 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 08:55:57,227 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-08 08:55:57,227 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:55:57,227 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,227 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:57,228 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 08:55:57,228 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:57,228 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,228 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-08 08:55:57,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:57,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:57,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:57,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:57,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:57,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:55:57,231 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:55:57,231 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-08 08:55:57,231 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:57,231 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:57,231 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,231 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:57,232 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,232 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,232 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:57,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:57,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:57,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:57,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:57,233 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:57,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:57,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,234 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:57,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,236 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:57,236 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:57,236 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 08:55:57,236 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:57,236 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:57,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:57,236 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:57,236 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,236 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,236 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,236 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:55:57,236 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:57,236 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:57,237 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:57,237 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,337 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:57,337 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:57,337 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:57,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:57,341 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:57,341 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:57,341 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:57,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:57,343 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,343 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 08:55:57,343 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:57,343 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 08:55:57,343 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 08:55:57,344 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,344 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,346 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,346 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:57,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:57,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:57,346 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:57,346 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:57,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,346 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:57,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,347 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,347 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 08:55:57,347 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:57,347 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 08:55:57,347 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:57,347 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 08:55:57,347 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:57,347 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:57,347 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:55:57,347 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. started... 2023-02-08 08:55:57,348 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 70af1f4bea4eb4fc084512e4e5840121 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:55:57,759 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512287ce72f4b92ae4e6691f4699fd7582fb8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512287ce72f4b92ae4e6691f4699fd7582fb8 2023-02-08 08:55:58,164 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/4ff8dcfdc6f04e92a54f3281e6e76cab 2023-02-08 08:55:58,168 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/4ff8dcfdc6f04e92a54f3281e6e76cab as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/4ff8dcfdc6f04e92a54f3281e6e76cab 2023-02-08 08:55:58,171 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/4ff8dcfdc6f04e92a54f3281e6e76cab, entries=1, sequenceid=21, filesize=4.9 K 2023-02-08 08:55:58,171 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 70af1f4bea4eb4fc084512e4e5840121 in 823ms, sequenceid=21, compaction requested=true 2023-02-08 08:55:58,171 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 70af1f4bea4eb4fc084512e4e5840121: 2023-02-08 08:55:58,172 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:55:58,172 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:58,172 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:58,172 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:58,172 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 08:55:58,172 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-08 08:55:58,173 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,173 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:58,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:58,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:58,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:58,173 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,174 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:58,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:58,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:58,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:58,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:58,175 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 08:55:58,175 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e01c832[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:58,175 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 08:55:58,175 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,176 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,176 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:58,176 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:58,176 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 08:55:58,176 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-08 08:55:58,176 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:55:58,177 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 08:55:58,177 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:58,177 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:58,177 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-08 08:55:58,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:58,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:58,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:58,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:58,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:58,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:58,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:58,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:58,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:55:58,179 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:55:58,179 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-08 08:55:58,179 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:58,179 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:58,179 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,179 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:58,182 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,182 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:58,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:58,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,182 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:58,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:58,183 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:58,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:58,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:58,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:58,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,184 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:58,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:58,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:58,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:58,188 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:58,188 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:58,188 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 08:55:58,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:58,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:58,188 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:58,188 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:58,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:58,188 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 08:55:58,188 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,189 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:58,189 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,189 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:55:58,189 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,189 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:58,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:58,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:58,189 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,289 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:58,289 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:58,289 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:58,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:58,293 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:58,293 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:58,293 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:58,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:58,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,295 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 08:55:58,295 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:58,295 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 08:55:58,295 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 08:55:58,295 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,295 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,296 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:58,296 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:58,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:58,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:58,297 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:58,297 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:58,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,297 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:58,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,297 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,298 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 08:55:58,298 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:58,298 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 08:55:58,298 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:58,298 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 08:55:58,298 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:58,298 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:58,298 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:55:58,298 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. started... 2023-02-08 08:55:58,298 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 70af1f4bea4eb4fc084512e4e5840121 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:55:58,714 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512316f1a34faec7040eaaeb6132212a0f7ac to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512316f1a34faec7040eaaeb6132212a0f7ac 2023-02-08 08:55:59,119 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/c9145cc7d68744d0b7133b9c5462c637 2023-02-08 08:55:59,124 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/c9145cc7d68744d0b7133b9c5462c637 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/c9145cc7d68744d0b7133b9c5462c637 2023-02-08 08:55:59,127 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/c9145cc7d68744d0b7133b9c5462c637, entries=1, sequenceid=25, filesize=4.9 K 2023-02-08 08:55:59,128 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 70af1f4bea4eb4fc084512e4e5840121 in 830ms, sequenceid=25, compaction requested=true 2023-02-08 08:55:59,128 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 70af1f4bea4eb4fc084512e4e5840121: 2023-02-08 08:55:59,128 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:55:59,128 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:55:59,128 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:55:59,129 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:59,129 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 08:55:59,129 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-08 08:55:59,130 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:59,130 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:59,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:59,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:59,130 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:59,130 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:55:59,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:59,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:59,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:59,131 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 08:55:59,132 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@13bfec77[Count = 0] remaining members to acquire global barrier 2023-02-08 08:55:59,132 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 08:55:59,132 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,134 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,134 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 08:55:59,134 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-08 08:55:59,134 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:59,134 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:55:59,134 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:55:59,135 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:59,135 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 08:55:59,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:59,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:59,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:59,135 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:59,135 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-08 08:55:59,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:59,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:59,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:59,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:59,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:59,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:55:59,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:55:59,137 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-08 08:55:59,137 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:55:59,137 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:55:59,137 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,137 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:55:59,138 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,139 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,139 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:59,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:55:59,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:55:59,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:55:59,139 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:59,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:59,139 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:59,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:55:59,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:59,140 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:59,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:55:59,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:59,142 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:59,142 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:59,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:59,142 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:55:59,142 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 08:55:59,142 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:55:59,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:59,142 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,143 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,143 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:59,143 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,142 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:55:59,143 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,143 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,143 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:55:59,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:55:59,143 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:55:59,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:55:59,243 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:55:59,243 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:59,243 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:59,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:55:59,247 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:55:59,247 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:55:59,248 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:55:59,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:55:59,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,249 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 08:55:59,249 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:59,249 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 08:55:59,249 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 08:55:59,250 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,250 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,252 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:55:59,252 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:59,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:55:59,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:55:59,252 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:55:59,252 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:55:59,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,253 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:55:59,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,253 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,253 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 08:55:59,253 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-08 08:55:59,253 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 08:55:59,253 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:55:59,253 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 08:55:59,253 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:55:59,253 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:55:59,253 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:55:59,254 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. started... 2023-02-08 08:55:59,254 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 70af1f4bea4eb4fc084512e4e5840121 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:55:59,664 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101e329b2217491439a86e3e982999be262 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101e329b2217491439a86e3e982999be262 2023-02-08 08:56:00,003 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 08:56:00,069 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/5a3352c1c6654e9abe60c8d6fac6488a 2023-02-08 08:56:00,073 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/5a3352c1c6654e9abe60c8d6fac6488a as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/5a3352c1c6654e9abe60c8d6fac6488a 2023-02-08 08:56:00,077 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/5a3352c1c6654e9abe60c8d6fac6488a, entries=1, sequenceid=29, filesize=4.9 K 2023-02-08 08:56:00,078 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 70af1f4bea4eb4fc084512e4e5840121 in 824ms, sequenceid=29, compaction requested=true 2023-02-08 08:56:00,078 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 70af1f4bea4eb4fc084512e4e5840121: 2023-02-08 08:56:00,078 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:56:00,078 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:56:00,078 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:56:00,078 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,078 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 08:56:00,078 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-08 08:56:00,079 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,079 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:00,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:00,080 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,080 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:56:00,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:00,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:00,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:00,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 08:56:00,081 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55611e45[Count = 0] remaining members to acquire global barrier 2023-02-08 08:56:00,081 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 08:56:00,081 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,082 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,083 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 08:56:00,083 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-08 08:56:00,083 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:56:00,083 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,083 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:56:00,084 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,084 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 08:56:00,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,084 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:56:00,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:00,084 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-08 08:56:00,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:00,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:00,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:00,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:00,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:56:00,086 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:56:00,086 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-08 08:56:00,086 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:56:00,086 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:56:00,086 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,086 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:56:00,089 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,089 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,089 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:00,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:00,089 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:56:00,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:00,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:56:00,090 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:56:00,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:00,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,091 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:00,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,093 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:56:00,093 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:56:00,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:56:00,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:56:00,093 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:56:00,093 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,093 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 08:56:00,093 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:56:00,093 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,093 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:56:00,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:56:00,093 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,093 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,093 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,093 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,093 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:56:00,093 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 08:56:00,093 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:56:00,194 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:56:00,194 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:00,194 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:00,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:56:00,198 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:00,198 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:00,199 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:56:00,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:56:00,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,200 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 08:56:00,200 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:56:00,200 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 08:56:00,200 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 08:56:00,201 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,201 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,202 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:56:00,202 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,202 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:56:00,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:56:00,202 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,202 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:56:00,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,203 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:56:00,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,203 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,203 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 08:56:00,203 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,203 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 08:56:00,203 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:56:00,204 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 08:56:00,204 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:56:00,204 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:56:00,204 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:56:00,204 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. started... 2023-02-08 08:56:00,204 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 70af1f4bea4eb4fc084512e4e5840121 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:56:00,215 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601032ca769fbda144970a8b92f21a6c4fdf8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601032ca769fbda144970a8b92f21a6c4fdf8 2023-02-08 08:56:00,620 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/d567f8f342264d7e99d17a4f8c471e57 2023-02-08 08:56:00,623 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/d567f8f342264d7e99d17a4f8c471e57 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/d567f8f342264d7e99d17a4f8c471e57 2023-02-08 08:56:00,626 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/d567f8f342264d7e99d17a4f8c471e57, entries=1, sequenceid=33, filesize=4.9 K 2023-02-08 08:56:00,627 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 70af1f4bea4eb4fc084512e4e5840121 in 423ms, sequenceid=33, compaction requested=true 2023-02-08 08:56:00,627 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 70af1f4bea4eb4fc084512e4e5840121: 2023-02-08 08:56:00,627 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:56:00,627 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:56:00,627 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:56:00,627 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,627 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 08:56:00,627 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-08 08:56:00,629 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,629 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:00,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:00,629 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,629 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:56:00,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:00,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:00,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:00,630 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 08:56:00,630 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@475c321a[Count = 0] remaining members to acquire global barrier 2023-02-08 08:56:00,630 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 08:56:00,631 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,633 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,633 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 08:56:00,633 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,633 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:56:00,633 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-08 08:56:00,633 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:56:00,634 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 08:56:00,634 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,635 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:56:00,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:00,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:00,635 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-08 08:56:00,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:00,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:00,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:00,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:56:00,637 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:56:00,637 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-08 08:56:00,637 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:56:00,637 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:56:00,637 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,637 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:56:00,638 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,638 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:00,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:00,638 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:56:00,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,638 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:00,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:56:00,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:56:00,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:00,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,639 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:00,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,645 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,645 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:56:00,645 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,645 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 08:56:00,645 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:56:00,645 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:56:00,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:56:00,645 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 08:56:00,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:56:00,645 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:56:00,645 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,645 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:56:00,645 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:56:00,645 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,645 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,645 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,646 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,646 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:56:00,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:56:00,745 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:56:00,746 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:00,746 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:00,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:56:00,749 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:00,749 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:00,750 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:56:00,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:56:00,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,751 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 08:56:00,751 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:56:00,751 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 08:56:00,751 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 08:56:00,752 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,752 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,753 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,753 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:56:00,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:56:00,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:56:00,753 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:00,753 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:56:00,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:56:00,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,754 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,754 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 08:56:00,754 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:00,754 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 08:56:00,754 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:56:00,754 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 08:56:00,755 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:56:00,755 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:56:00,755 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:56:00,755 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. started... 2023-02-08 08:56:00,755 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 70af1f4bea4eb4fc084512e4e5840121 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:56:01,167 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907012ab1c0d2684a54bbccbc55f0070a8f to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20150907012ab1c0d2684a54bbccbc55f0070a8f 2023-02-08 08:56:01,572 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/df4bf00ea3df4d969914c5fa38320792 2023-02-08 08:56:01,575 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/df4bf00ea3df4d969914c5fa38320792 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/df4bf00ea3df4d969914c5fa38320792 2023-02-08 08:56:01,578 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/df4bf00ea3df4d969914c5fa38320792, entries=1, sequenceid=37, filesize=4.9 K 2023-02-08 08:56:01,579 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 70af1f4bea4eb4fc084512e4e5840121 in 824ms, sequenceid=37, compaction requested=true 2023-02-08 08:56:01,579 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 70af1f4bea4eb4fc084512e4e5840121: 2023-02-08 08:56:01,579 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:56:01,579 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:56:01,579 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:56:01,579 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:01,579 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 08:56:01,579 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-08 08:56:01,580 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,580 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:01,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:01,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:01,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:01,581 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,581 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:56:01,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:01,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:01,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:01,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:01,582 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 08:56:01,582 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@37ec229b[Count = 0] remaining members to acquire global barrier 2023-02-08 08:56:01,582 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 08:56:01,582 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,584 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,584 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 08:56:01,585 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-08 08:56:01,585 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:01,585 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:56:01,585 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:56:01,586 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 08:56:01,586 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:01,586 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:56:01,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:01,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:01,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:01,586 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-08 08:56:01,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:01,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:01,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:01,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:01,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:01,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:56:01,588 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:56:01,588 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-08 08:56:01,588 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:56:01,588 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:56:01,588 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,588 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:56:01,589 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,589 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,589 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:56:01,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:01,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:01,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,590 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:01,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:56:01,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:01,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:56:01,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:01,591 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:01,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:01,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:01,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:01,592 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:01,593 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,593 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,593 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:56:01,593 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:56:01,593 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:56:01,593 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:01,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:56:01,593 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 08:56:01,593 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:56:01,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:56:01,593 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-08 08:56:01,593 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,593 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,593 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,593 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:56:01,593 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:56:01,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:56:01,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:56:01,693 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:56:01,694 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:01,694 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:01,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:56:01,697 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:01,697 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:01,698 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:56:01,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-08 08:56:01,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,699 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 08:56:01,700 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:56:01,700 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-08 08:56:01,700 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-08 08:56:01,700 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,700 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,702 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:01,702 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:56:01,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:56:01,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:56:01,702 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:01,702 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:56:01,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,702 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:56:01,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,703 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,703 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-08 08:56:01,703 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:01,703 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-08 08:56:01,703 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:56:01,703 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-08 08:56:01,703 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-08 08:56:01,703 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:56:01,703 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:56:01,703 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. started... 2023-02-08 08:56:01,703 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 70af1f4bea4eb4fc084512e4e5840121 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-08 08:56:02,115 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120cc4277dc5aa545108dcba4f943772cd3 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120cc4277dc5aa545108dcba4f943772cd3 2023-02-08 08:56:02,119 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/fb96ef9b5bee42e384f47ba1400d67d6 2023-02-08 08:56:02,123 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/fb96ef9b5bee42e384f47ba1400d67d6 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/fb96ef9b5bee42e384f47ba1400d67d6 2023-02-08 08:56:02,126 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/fb96ef9b5bee42e384f47ba1400d67d6, entries=1, sequenceid=41, filesize=4.9 K 2023-02-08 08:56:02,127 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 70af1f4bea4eb4fc084512e4e5840121 in 424ms, sequenceid=41, compaction requested=true 2023-02-08 08:56:02,127 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 70af1f4bea4eb4fc084512e4e5840121: 2023-02-08 08:56:02,127 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:56:02,127 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-08 08:56:02,127 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-08 08:56:02,127 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:02,127 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-08 08:56:02,127 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-08 08:56:02,205 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:02,205 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:02,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:02,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:02,206 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,206 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:56:02,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:02,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:02,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:02,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:02,207 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-08 08:56:02,207 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5b5cbc45[Count = 0] remaining members to acquire global barrier 2023-02-08 08:56:02,207 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-08 08:56:02,207 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,236 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,236 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,236 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-08 08:56:02,236 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:02,236 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:56:02,236 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-08 08:56:02,236 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:56:02,244 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-08 08:56:02,244 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:56:02,244 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:02,244 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-08 08:56:02,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:02,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:02,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:02,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:02,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:02,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:02,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:02,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:02,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:56:02,246 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-08 08:56:02,246 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-08 08:56:02,246 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:56:02,246 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:56:02,246 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,246 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:56:02,251 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,251 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,251 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:56:02,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:02,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:02,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:56:02,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:56:02,252 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:02,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:02,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:02,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,253 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:02,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:02,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:02,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:02,260 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:56:02,260 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:02,260 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:56:02,260 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,260 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-08 08:56:02,260 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:56:02,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:56:02,260 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,260 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:56:02,260 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:02,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:56:02,260 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-08 08:56:02,260 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,261 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:56:02,261 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,261 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:56:02,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:56:02,361 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:56:02,361 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:02,361 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:02,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:56:02,363 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37869] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-02-08 08:56:02,364 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 08:56:02,365 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 08:56:02,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:56:02,366 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675847162366 2023-02-08 08:56:02,367 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-08 08:56:02,368 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-02-08 08:56:02,368 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 08:56:02,368 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] zookeeper.ReadOnlyZKClient(139): Connect 0x1ef7f9d3 to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:56:02,372 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6456c338, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:56:02,372 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 08:56:03,198 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103c70545769d7743949960479bbddc7a07 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103c70545769d7743949960479bbddc7a07 2023-02-08 08:56:03,199 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 08:56:03,211 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:56:03,214 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46340, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:56:03,218 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:03,218 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:03,219 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 08:56:03,220 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33890, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 08:56:03,228 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/7f7588d3963d446cb82d6a0143f2a0e9 first=Optional[r5] last=Optional[r6] 2023-02-08 08:56:03,228 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/7f7588d3963d446cb82d6a0143f2a0e9}] 2023-02-08 08:56:03,232 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/7f7588d3963d446cb82d6a0143f2a0e9 for inclusion in 70af1f4bea4eb4fc084512e4e5840121/family1 2023-02-08 08:56:03,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-02-08 08:56:03,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 08:56:03,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for 70af1f4bea4eb4fc084512e4e5840121: 2023-02-08 08:56:03,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/7f7588d3963d446cb82d6a0143f2a0e9 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithMonthlyPolicy__3dn8lpmk5o9rinfsna8secppuuf5racnqeq67iqdsa6f2ccdrgl1k2qbumced3e0/family1/7f7588d3963d446cb82d6a0143f2a0e9 2023-02-08 08:56:03,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithMonthlyPolicy__3dn8lpmk5o9rinfsna8secppuuf5racnqeq67iqdsa6f2ccdrgl1k2qbumced3e0/family1/7f7588d3963d446cb82d6a0143f2a0e9 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/574ddbf2ea854b359b3c1fee09f917f6_SeqId_44_ 2023-02-08 08:56:03,237 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithMonthlyPolicy__3dn8lpmk5o9rinfsna8secppuuf5racnqeq67iqdsa6f2ccdrgl1k2qbumced3e0/family1/7f7588d3963d446cb82d6a0143f2a0e9 into 70af1f4bea4eb4fc084512e4e5840121/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/574ddbf2ea854b359b3c1fee09f917f6_SeqId_44_ - updating store file list. 2023-02-08 08:56:03,240 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/574ddbf2ea854b359b3c1fee09f917f6_SeqId_44_ into 70af1f4bea4eb4fc084512e4e5840121/family1 2023-02-08 08:56:03,240 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithMonthlyPolicy__3dn8lpmk5o9rinfsna8secppuuf5racnqeq67iqdsa6f2ccdrgl1k2qbumced3e0/family1/7f7588d3963d446cb82d6a0143f2a0e9 into 70af1f4bea4eb4fc084512e4e5840121/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/574ddbf2ea854b359b3c1fee09f917f6_SeqId_44_) 2023-02-08 08:56:03,240 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithMonthlyPolicy__3dn8lpmk5o9rinfsna8secppuuf5racnqeq67iqdsa6f2ccdrgl1k2qbumced3e0/family1/7f7588d3963d446cb82d6a0143f2a0e9 2023-02-08 08:56:03,240 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-08 08:56:03,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-08 08:56:03,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(1912): 70af1f4bea4eb4fc084512e4e5840121/family1 is initiating minor compaction 2023-02-08 08:56:03,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3be227df; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:56:03,243 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 70af1f4bea4eb4fc084512e4e5840121/family1 in testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:56:03,243 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/05135a0a28bf4168b73d32fb60984230, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/4e4cfc6cfc7b458f8cd951605f739daf, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/7e94dc57529c41fea673a8e7604137cd, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/4ff8dcfdc6f04e92a54f3281e6e76cab, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/c9145cc7d68744d0b7133b9c5462c637, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/5a3352c1c6654e9abe60c8d6fac6488a, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/d567f8f342264d7e99d17a4f8c471e57, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/df4bf00ea3df4d969914c5fa38320792, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/fb96ef9b5bee42e384f47ba1400d67d6, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/574ddbf2ea854b359b3c1fee09f917f6_SeqId_44_] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp, totalSize=49.7 K 2023-02-08 08:56:03,243 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 05135a0a28bf4168b73d32fb60984230, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-08 08:56:03,243 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 4e4cfc6cfc7b458f8cd951605f739daf, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-08 08:56:03,243 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 7e94dc57529c41fea673a8e7604137cd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-08 08:56:03,243 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 4ff8dcfdc6f04e92a54f3281e6e76cab, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-08 08:56:03,243 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting c9145cc7d68744d0b7133b9c5462c637, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-08 08:56:03,243 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 5a3352c1c6654e9abe60c8d6fac6488a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-08 08:56:03,243 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting d567f8f342264d7e99d17a4f8c471e57, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-08 08:56:03,243 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting df4bf00ea3df4d969914c5fa38320792, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-08 08:56:03,243 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting fb96ef9b5bee42e384f47ba1400d67d6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-08 08:56:03,243 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 574ddbf2ea854b359b3c1fee09f917f6_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-08 08:56:03,245 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 08:56:03,246 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:56:03,247 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101e329b2217491439a86e3e982999be262 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160101e329b2217491439a86e3e982999be262 2023-02-08 08:56:03,260 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601032ca769fbda144970a8b92f21a6c4fdf8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601032ca769fbda144970a8b92f21a6c4fdf8 2023-02-08 08:56:03,260 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-02-08 08:56:03,260 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-02-08 08:56:03,260 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 08:56:03,260 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 08:56:03,271 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] throttle.PressureAwareThroughputController(145): 70af1f4bea4eb4fc084512e4e5840121#family1#compaction#165 average throughput is 1.37 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-08 08:56:03,691 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/52abb63ce3d64c1aa905e6c96e643e42 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/52abb63ce3d64c1aa905e6c96e643e42 2023-02-08 08:56:03,696 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 70af1f4bea4eb4fc084512e4e5840121/family1 of 70af1f4bea4eb4fc084512e4e5840121 into 52abb63ce3d64c1aa905e6c96e643e42(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:56:03,696 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 70af1f4bea4eb4fc084512e4e5840121: 2023-02-08 08:56:03,696 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1675846563243; duration=0sec 2023-02-08 08:56:03,696 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:56:03,713 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/05135a0a28bf4168b73d32fb60984230, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/4e4cfc6cfc7b458f8cd951605f739daf, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/7e94dc57529c41fea673a8e7604137cd, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/4ff8dcfdc6f04e92a54f3281e6e76cab, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/c9145cc7d68744d0b7133b9c5462c637, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/5a3352c1c6654e9abe60c8d6fac6488a, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/d567f8f342264d7e99d17a4f8c471e57, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/df4bf00ea3df4d969914c5fa38320792, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/fb96ef9b5bee42e384f47ba1400d67d6, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/574ddbf2ea854b359b3c1fee09f917f6_SeqId_44_] to archive 2023-02-08 08:56:03,714 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:56:03,715 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/05135a0a28bf4168b73d32fb60984230 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/05135a0a28bf4168b73d32fb60984230 2023-02-08 08:56:03,717 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/4e4cfc6cfc7b458f8cd951605f739daf to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/4e4cfc6cfc7b458f8cd951605f739daf 2023-02-08 08:56:03,718 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/7e94dc57529c41fea673a8e7604137cd to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/7e94dc57529c41fea673a8e7604137cd 2023-02-08 08:56:03,719 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/4ff8dcfdc6f04e92a54f3281e6e76cab to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/4ff8dcfdc6f04e92a54f3281e6e76cab 2023-02-08 08:56:03,720 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/c9145cc7d68744d0b7133b9c5462c637 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/c9145cc7d68744d0b7133b9c5462c637 2023-02-08 08:56:03,722 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/5a3352c1c6654e9abe60c8d6fac6488a to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/5a3352c1c6654e9abe60c8d6fac6488a 2023-02-08 08:56:03,723 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/d567f8f342264d7e99d17a4f8c471e57 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/d567f8f342264d7e99d17a4f8c471e57 2023-02-08 08:56:03,724 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/df4bf00ea3df4d969914c5fa38320792 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/df4bf00ea3df4d969914c5fa38320792 2023-02-08 08:56:03,725 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/fb96ef9b5bee42e384f47ba1400d67d6 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/fb96ef9b5bee42e384f47ba1400d67d6 2023-02-08 08:56:03,726 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/574ddbf2ea854b359b3c1fee09f917f6_SeqId_44_ to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/574ddbf2ea854b359b3c1fee09f917f6_SeqId_44_ 2023-02-08 08:56:04,109 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231b2ba274d2e624331afced431bbe7d65c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231b2ba274d2e624331afced431bbe7d65c 2023-02-08 08:56:04,110 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 08:56:04,133 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:04,133 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:04,141 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/2c02ebc6f69d443489aada24ae2ad4c7 first=Optional[r1] last=Optional[r4] 2023-02-08 08:56:04,142 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/2c02ebc6f69d443489aada24ae2ad4c7}] 2023-02-08 08:56:04,144 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/2c02ebc6f69d443489aada24ae2ad4c7 for inclusion in 70af1f4bea4eb4fc084512e4e5840121/family1 2023-02-08 08:56:04,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-02-08 08:56:04,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 08:56:04,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for 70af1f4bea4eb4fc084512e4e5840121: 2023-02-08 08:56:04,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/2c02ebc6f69d443489aada24ae2ad4c7 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithMonthlyPolicy__qdod5po6de5fecm9ccppk6v96v80kj00okgoscsdkmrou1rfgo9lalal567jnr0v/family1/2c02ebc6f69d443489aada24ae2ad4c7 2023-02-08 08:56:04,150 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithMonthlyPolicy__qdod5po6de5fecm9ccppk6v96v80kj00okgoscsdkmrou1rfgo9lalal567jnr0v/family1/2c02ebc6f69d443489aada24ae2ad4c7 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/08dfffad7b764c90a9ec0823ff825cc7_SeqId_47_ 2023-02-08 08:56:04,150 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithMonthlyPolicy__qdod5po6de5fecm9ccppk6v96v80kj00okgoscsdkmrou1rfgo9lalal567jnr0v/family1/2c02ebc6f69d443489aada24ae2ad4c7 into 70af1f4bea4eb4fc084512e4e5840121/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/08dfffad7b764c90a9ec0823ff825cc7_SeqId_47_ - updating store file list. 2023-02-08 08:56:04,154 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/08dfffad7b764c90a9ec0823ff825cc7_SeqId_47_ into 70af1f4bea4eb4fc084512e4e5840121/family1 2023-02-08 08:56:04,154 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithMonthlyPolicy__qdod5po6de5fecm9ccppk6v96v80kj00okgoscsdkmrou1rfgo9lalal567jnr0v/family1/2c02ebc6f69d443489aada24ae2ad4c7 into 70af1f4bea4eb4fc084512e4e5840121/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/08dfffad7b764c90a9ec0823ff825cc7_SeqId_47_) 2023-02-08 08:56:04,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithMonthlyPolicy__qdod5po6de5fecm9ccppk6v96v80kj00okgoscsdkmrou1rfgo9lalal567jnr0v/family1/2c02ebc6f69d443489aada24ae2ad4c7 2023-02-08 08:56:04,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-08 08:56:04,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-08 08:56:04,156 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(1912): 70af1f4bea4eb4fc084512e4e5840121/family1 is initiating minor compaction (all files) 2023-02-08 08:56:04,156 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@31eac279; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:56:04,156 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 70af1f4bea4eb4fc084512e4e5840121/family1 in testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:56:04,156 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/2b8ae81853414d7797c207b766994168, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/52abb63ce3d64c1aa905e6c96e643e42, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/08dfffad7b764c90a9ec0823ff825cc7_SeqId_47_] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp, totalSize=178.1 K 2023-02-08 08:56:04,156 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 2b8ae81853414d7797c207b766994168, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878836535 2023-02-08 08:56:04,157 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 52abb63ce3d64c1aa905e6c96e643e42, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621236535 2023-02-08 08:56:04,157 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] compactions.Compactor(207): Compacting 08dfffad7b764c90a9ec0823ff825cc7_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965236535 2023-02-08 08:56:04,157 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 08:56:04,158 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:56:04,159 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512011aaf9f12132548cdaf8a35903df2baa6 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512011aaf9f12132548cdaf8a35903df2baa6 2023-02-08 08:56:04,161 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205ab29dcc763db4adab7abddfcc27ea3b6 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205ab29dcc763db4adab7abddfcc27ea3b6 2023-02-08 08:56:04,162 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512287ce72f4b92ae4e6691f4699fd7582fb8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512287ce72f4b92ae4e6691f4699fd7582fb8 2023-02-08 08:56:04,166 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512316f1a34faec7040eaaeb6132212a0f7ac to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512316f1a34faec7040eaaeb6132212a0f7ac 2023-02-08 08:56:04,167 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-02-08 08:56:04,167 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-02-08 08:56:04,167 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 08:56:04,167 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 08:56:04,175 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] throttle.PressureAwareThroughputController(145): 70af1f4bea4eb4fc084512e4e5840121#family1#compaction#166 average throughput is 25.93 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 53.85 MB/second 2023-02-08 08:56:05,019 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511309f86256b227f486cb2f857b75e5a47b7 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511309f86256b227f486cb2f857b75e5a47b7 2023-02-08 08:56:05,020 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 08:56:05,035 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:05,036 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:05,043 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b3aab57f607841ed9e2ad43018c65c5c first=Optional[r00] last=Optional[r8] 2023-02-08 08:56:05,044 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b3aab57f607841ed9e2ad43018c65c5c}] 2023-02-08 08:56:05,046 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b3aab57f607841ed9e2ad43018c65c5c for inclusion in 70af1f4bea4eb4fc084512e4e5840121/family1 2023-02-08 08:56:05,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-02-08 08:56:05,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last= 2023-02-08 08:56:05,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for 70af1f4bea4eb4fc084512e4e5840121: 2023-02-08 08:56:05,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/b3aab57f607841ed9e2ad43018c65c5c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithMonthlyPolicy__4bhjfnqnho2rj268qfaehe7iavgt6nqesvh7ol8v4d7j4at6sk6h48feu8ocedup/family1/b3aab57f607841ed9e2ad43018c65c5c 2023-02-08 08:56:05,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithMonthlyPolicy__4bhjfnqnho2rj268qfaehe7iavgt6nqesvh7ol8v4d7j4at6sk6h48feu8ocedup/family1/b3aab57f607841ed9e2ad43018c65c5c as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/2d75e34aa21d40378638b23ecfd3b7d5_SeqId_49_ 2023-02-08 08:56:05,050 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithMonthlyPolicy__4bhjfnqnho2rj268qfaehe7iavgt6nqesvh7ol8v4d7j4at6sk6h48feu8ocedup/family1/b3aab57f607841ed9e2ad43018c65c5c into 70af1f4bea4eb4fc084512e4e5840121/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/2d75e34aa21d40378638b23ecfd3b7d5_SeqId_49_ - updating store file list. 2023-02-08 08:56:05,053 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/2d75e34aa21d40378638b23ecfd3b7d5_SeqId_49_ into 70af1f4bea4eb4fc084512e4e5840121/family1 2023-02-08 08:56:05,053 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithMonthlyPolicy__4bhjfnqnho2rj268qfaehe7iavgt6nqesvh7ol8v4d7j4at6sk6h48feu8ocedup/family1/b3aab57f607841ed9e2ad43018c65c5c into 70af1f4bea4eb4fc084512e4e5840121/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/2d75e34aa21d40378638b23ecfd3b7d5_SeqId_49_) 2023-02-08 08:56:05,053 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMinorCompactionWithMonthlyPolicy__4bhjfnqnho2rj268qfaehe7iavgt6nqesvh7ol8v4d7j4at6sk6h48feu8ocedup/family1/b3aab57f607841ed9e2ad43018c65c5c 2023-02-08 08:56:05,055 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 08:56:05,056 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:56:05,057 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120cc4277dc5aa545108dcba4f943772cd3 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120cc4277dc5aa545108dcba4f943772cd3 2023-02-08 08:56:05,058 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511283fd2061d56af4272a57c48ace01928e6 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511283fd2061d56af4272a57c48ace01928e6 2023-02-08 08:56:05,059 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130a39ecbb027f6437eb9d40930132197eb to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130a39ecbb027f6437eb9d40930132197eb 2023-02-08 08:56:05,059 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-02-08 08:56:05,059 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-08 08:56:05,059 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 08:56:05,059 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 08:56:05,059 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-08 08:56:05,059 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-08 08:56:05,060 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:56:05,062 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-08 08:56:05,063 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.6980 sec 2023-02-08 08:56:05,250 INFO [Listener at localhost/40763] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=257 (was 248) - Thread LEAK? -, OpenFileDescriptor=645 (was 621) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=130 (was 120) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 168) - ProcessCount LEAK? -, AvailableMemoryMB=4247 (was 4179) - AvailableMemoryMB LEAK? - 2023-02-08 08:56:05,261 INFO [Listener at localhost/40763] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=257, OpenFileDescriptor=645, MaxFileDescriptor=60000, SystemLoadAverage=130, ProcessCount=170, AvailableMemoryMB=4247 2023-02-08 08:56:05,280 DEBUG [Listener at localhost/40763] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-02-08 08:56:05,303 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:05,303 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:05,335 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10h\xCB\x89\xB7f\x9B\x1B\xABuz\xD8\xF5\xD4\xF8Lm"\x10\x173\x13\xF0B\xA0\x0E\x08\x0A\x1F\xA5\x0ExAtd*\x10%\x94\xE4\x99\xB4\xF5\xA5\xB3\x8C\xBC?+\xECC\x86\x162\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-08 08:56:05,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-02-08 08:56:05,338 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-02-08 08:56:05,338 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-02-08 08:56:05,338 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:05,338 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:05,338 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-08 08:56:05,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-08 08:56:05,340 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f 2023-02-08 08:56:05,340 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa 2023-02-08 08:56:05,340 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885 2023-02-08 08:56:05,341 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f empty. 2023-02-08 08:56:05,341 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885 empty. 2023-02-08 08:56:05,341 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa empty. 2023-02-08 08:56:05,341 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f 2023-02-08 08:56:05,341 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885 2023-02-08 08:56:05,341 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa 2023-02-08 08:56:05,341 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-02-08 08:56:05,412 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/.tmp/family1/f2c49c2858b54d12ac83707a34b6531c as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/f2c49c2858b54d12ac83707a34b6531c 2023-02-08 08:56:05,416 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 70af1f4bea4eb4fc084512e4e5840121/family1 of 70af1f4bea4eb4fc084512e4e5840121 into f2c49c2858b54d12ac83707a34b6531c(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 08:56:05,416 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 70af1f4bea4eb4fc084512e4e5840121: 2023-02-08 08:56:05,416 INFO [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1675846564156; duration=1sec 2023-02-08 08:56:05,416 DEBUG [RS:0;jenkins-hbase4:42239-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-08 08:56:05,439 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:05,439 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:05,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-08 08:56:05,513 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/2b8ae81853414d7797c207b766994168, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/52abb63ce3d64c1aa905e6c96e643e42, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/08dfffad7b764c90a9ec0823ff825cc7_SeqId_47_] to archive 2023-02-08 08:56:05,514 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:56:05,515 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/2b8ae81853414d7797c207b766994168 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/2b8ae81853414d7797c207b766994168 2023-02-08 08:56:05,516 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/52abb63ce3d64c1aa905e6c96e643e42 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/52abb63ce3d64c1aa905e6c96e643e42 2023-02-08 08:56:05,517 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/08dfffad7b764c90a9ec0823ff825cc7_SeqId_47_ to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/family1/08dfffad7b764c90a9ec0823ff825cc7_SeqId_47_ 2023-02-08 08:56:05,640 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:05,640 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:05,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-08 08:56:05,748 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-02-08 08:56:05,749 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7671): creating {ENCODED => e0c4a41d57aa926b6b63e3ff74002885, NAME => 'testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10h\xCB\x89\xB7f\x9B\x1B\xABuz\xD8\xF5\xD4\xF8Lm"\x10\x173\x13\xF0B\xA0\x0E\x08\x0A\x1F\xA5\x0ExAtd*\x10%\x94\xE4\x99\xB4\xF5\xA5\xB3\x8C\xBC?+\xECC\x86\x162\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp 2023-02-08 08:56:05,750 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7671): creating {ENCODED => cde43f550d582d55f69773dc9ed101fa, NAME => 'testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10h\xCB\x89\xB7f\x9B\x1B\xABuz\xD8\xF5\xD4\xF8Lm"\x10\x173\x13\xF0B\xA0\x0E\x08\x0A\x1F\xA5\x0ExAtd*\x10%\x94\xE4\x99\xB4\xF5\xA5\xB3\x8C\xBC?+\xECC\x86\x162\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp 2023-02-08 08:56:05,750 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7671): creating {ENCODED => b9a58ce7226f8c74217cc9288a09172f, NAME => 'testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10h\xCB\x89\xB7f\x9B\x1B\xABuz\xD8\xF5\xD4\xF8Lm"\x10\x173\x13\xF0B\xA0\x0E\x08\x0A\x1F\xA5\x0ExAtd*\x10%\x94\xE4\x99\xB4\xF5\xA5\xB3\x8C\xBC?+\xECC\x86\x162\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/.tmp 2023-02-08 08:56:05,941 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:05,941 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:05,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-08 08:56:06,157 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:56:06,157 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:56:06,157 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1603): Closing b9a58ce7226f8c74217cc9288a09172f, disabling compactions & flushes 2023-02-08 08:56:06,157 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1603): Closing e0c4a41d57aa926b6b63e3ff74002885, disabling compactions & flushes 2023-02-08 08:56:06,157 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:06,157 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:06,157 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:06,157 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:06,157 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:56:06,157 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. after waiting 0 ms 2023-02-08 08:56:06,157 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. after waiting 0 ms 2023-02-08 08:56:06,157 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:06,157 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:06,157 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:06,157 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1603): Closing cde43f550d582d55f69773dc9ed101fa, disabling compactions & flushes 2023-02-08 08:56:06,158 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1557): Region close journal for e0c4a41d57aa926b6b63e3ff74002885: Waiting for close lock at 1675846566157Disabling compacts and flushes for region at 1675846566157Disabling writes for close at 1675846566157Writing region close event to WAL at 1675846566157Closed at 1675846566157 2023-02-08 08:56:06,157 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:06,158 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:06,158 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:06,158 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1557): Region close journal for b9a58ce7226f8c74217cc9288a09172f: Waiting for close lock at 1675846566157Disabling compacts and flushes for region at 1675846566157Disabling writes for close at 1675846566157Writing region close event to WAL at 1675846566157Closed at 1675846566157 2023-02-08 08:56:06,158 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. after waiting 0 ms 2023-02-08 08:56:06,158 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:06,158 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:06,158 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1557): Region close journal for cde43f550d582d55f69773dc9ed101fa: Waiting for close lock at 1675846566157Disabling compacts and flushes for region at 1675846566157Disabling writes for close at 1675846566158 (+1 ms)Writing region close event to WAL at 1675846566158Closed at 1675846566158 2023-02-08 08:56:06,160 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-02-08 08:56:06,160 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675846566160"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846566160"}]},"ts":"1675846566160"} 2023-02-08 08:56:06,161 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675846566160"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846566160"}]},"ts":"1675846566160"} 2023-02-08 08:56:06,161 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846566160"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1675846566160"}]},"ts":"1675846566160"} 2023-02-08 08:56:06,162 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-08 08:56:06,163 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-08 08:56:06,163 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846566163"}]},"ts":"1675846566163"} 2023-02-08 08:56:06,164 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-02-08 08:56:06,166 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=b9a58ce7226f8c74217cc9288a09172f, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=cde43f550d582d55f69773dc9ed101fa, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=e0c4a41d57aa926b6b63e3ff74002885, ASSIGN}] 2023-02-08 08:56:06,168 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=b9a58ce7226f8c74217cc9288a09172f, ASSIGN 2023-02-08 08:56:06,168 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=cde43f550d582d55f69773dc9ed101fa, ASSIGN 2023-02-08 08:56:06,168 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=e0c4a41d57aa926b6b63e3ff74002885, ASSIGN 2023-02-08 08:56:06,168 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=b9a58ce7226f8c74217cc9288a09172f, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=false 2023-02-08 08:56:06,169 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=cde43f550d582d55f69773dc9ed101fa, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=false 2023-02-08 08:56:06,169 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=e0c4a41d57aa926b6b63e3ff74002885, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42239,1675846413348; forceNewPlan=false, retain=false 2023-02-08 08:56:06,320 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=b9a58ce7226f8c74217cc9288a09172f, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:06,320 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=cde43f550d582d55f69773dc9ed101fa, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:06,320 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675846566320"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846566320"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846566320"}]},"ts":"1675846566320"} 2023-02-08 08:56:06,320 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=e0c4a41d57aa926b6b63e3ff74002885, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:06,320 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846566320"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846566320"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846566320"}]},"ts":"1675846566320"} 2023-02-08 08:56:06,321 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675846566320"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1675846566320"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1675846566320"}]},"ts":"1675846566320"} 2023-02-08 08:56:06,322 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=117, state=RUNNABLE; OpenRegionProcedure b9a58ce7226f8c74217cc9288a09172f, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:56:06,322 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=118, state=RUNNABLE; OpenRegionProcedure cde43f550d582d55f69773dc9ed101fa, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:56:06,323 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=119, state=RUNNABLE; OpenRegionProcedure e0c4a41d57aa926b6b63e3ff74002885, server=jenkins-hbase4.apache.org,42239,1675846413348}] 2023-02-08 08:56:06,442 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:06,442 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:06,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-08 08:56:06,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:06,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b9a58ce7226f8c74217cc9288a09172f, NAME => 'testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f.', STARTKEY => '', ENDKEY => '1'} 2023-02-08 08:56:06,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin b9a58ce7226f8c74217cc9288a09172f 2023-02-08 08:56:06,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:56:06,477 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b9a58ce7226f8c74217cc9288a09172f 2023-02-08 08:56:06,478 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b9a58ce7226f8c74217cc9288a09172f 2023-02-08 08:56:06,498 INFO [StoreOpener-b9a58ce7226f8c74217cc9288a09172f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region b9a58ce7226f8c74217cc9288a09172f 2023-02-08 08:56:06,500 DEBUG [StoreOpener-b9a58ce7226f8c74217cc9288a09172f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1 2023-02-08 08:56:06,500 DEBUG [StoreOpener-b9a58ce7226f8c74217cc9288a09172f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1 2023-02-08 08:56:06,500 INFO [StoreOpener-b9a58ce7226f8c74217cc9288a09172f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b9a58ce7226f8c74217cc9288a09172f columnFamilyName family1 2023-02-08 08:56:06,501 INFO [StoreOpener-b9a58ce7226f8c74217cc9288a09172f-1] regionserver.HStore(310): Store=b9a58ce7226f8c74217cc9288a09172f/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:56:06,502 INFO [StoreOpener-b9a58ce7226f8c74217cc9288a09172f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region b9a58ce7226f8c74217cc9288a09172f 2023-02-08 08:56:06,503 DEBUG [StoreOpener-b9a58ce7226f8c74217cc9288a09172f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2 2023-02-08 08:56:06,503 DEBUG [StoreOpener-b9a58ce7226f8c74217cc9288a09172f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2 2023-02-08 08:56:06,503 INFO [StoreOpener-b9a58ce7226f8c74217cc9288a09172f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b9a58ce7226f8c74217cc9288a09172f columnFamilyName family2 2023-02-08 08:56:06,504 INFO [StoreOpener-b9a58ce7226f8c74217cc9288a09172f-1] regionserver.HStore(310): Store=b9a58ce7226f8c74217cc9288a09172f/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:56:06,505 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f 2023-02-08 08:56:06,505 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f 2023-02-08 08:56:06,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:56:06,507 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b9a58ce7226f8c74217cc9288a09172f 2023-02-08 08:56:06,509 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:56:06,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b9a58ce7226f8c74217cc9288a09172f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63935168, jitterRate=-0.04729175567626953}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:56:06,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b9a58ce7226f8c74217cc9288a09172f: Running coprocessor pre-open hook at 1675846566478Writing region info on filesystem at 1675846566478Initializing all the Stores at 1675846566478Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10h\xCB\x89\xB7f\x9B\x1B\xABuz\xD8\xF5\xD4\xF8Lm"\x10\x173\x13\xF0B\xA0\x0E\x08\x0A\x1F\xA5\x0ExAtd*\x10%\x94\xE4\x99\xB4\xF5\xA5\xB3\x8C\xBC?+\xECC\x86\x162\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846566478Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846566478Cleaning up temporary data from old regions at 1675846566505 (+27 ms)Cleaning up detritus from prior splits at 1675846566506 (+1 ms)Running coprocessor post-open hooks at 1675846566510 (+4 ms)Region opened successfully at 1675846566510 2023-02-08 08:56:06,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f., pid=120, masterSystemTime=1675846566473 2023-02-08 08:56:06,511 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:06,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:06,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:06,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => cde43f550d582d55f69773dc9ed101fa, NAME => 'testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa.', STARTKEY => '1', ENDKEY => '2'} 2023-02-08 08:56:06,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin cde43f550d582d55f69773dc9ed101fa 2023-02-08 08:56:06,512 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=b9a58ce7226f8c74217cc9288a09172f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:06,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:56:06,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for cde43f550d582d55f69773dc9ed101fa 2023-02-08 08:56:06,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for cde43f550d582d55f69773dc9ed101fa 2023-02-08 08:56:06,512 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675846566512"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846566512"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846566512"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846566512"}]},"ts":"1675846566512"} 2023-02-08 08:56:06,515 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=117 2023-02-08 08:56:06,515 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=117, state=SUCCESS; OpenRegionProcedure b9a58ce7226f8c74217cc9288a09172f, server=jenkins-hbase4.apache.org,42239,1675846413348 in 191 msec 2023-02-08 08:56:06,516 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=b9a58ce7226f8c74217cc9288a09172f, ASSIGN in 349 msec 2023-02-08 08:56:06,532 INFO [StoreOpener-cde43f550d582d55f69773dc9ed101fa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region cde43f550d582d55f69773dc9ed101fa 2023-02-08 08:56:06,533 DEBUG [StoreOpener-cde43f550d582d55f69773dc9ed101fa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1 2023-02-08 08:56:06,534 DEBUG [StoreOpener-cde43f550d582d55f69773dc9ed101fa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1 2023-02-08 08:56:06,534 INFO [StoreOpener-cde43f550d582d55f69773dc9ed101fa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cde43f550d582d55f69773dc9ed101fa columnFamilyName family1 2023-02-08 08:56:06,535 INFO [StoreOpener-cde43f550d582d55f69773dc9ed101fa-1] regionserver.HStore(310): Store=cde43f550d582d55f69773dc9ed101fa/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:56:06,535 INFO [StoreOpener-cde43f550d582d55f69773dc9ed101fa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region cde43f550d582d55f69773dc9ed101fa 2023-02-08 08:56:06,536 DEBUG [StoreOpener-cde43f550d582d55f69773dc9ed101fa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2 2023-02-08 08:56:06,536 DEBUG [StoreOpener-cde43f550d582d55f69773dc9ed101fa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2 2023-02-08 08:56:06,536 INFO [StoreOpener-cde43f550d582d55f69773dc9ed101fa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cde43f550d582d55f69773dc9ed101fa columnFamilyName family2 2023-02-08 08:56:06,537 INFO [StoreOpener-cde43f550d582d55f69773dc9ed101fa-1] regionserver.HStore(310): Store=cde43f550d582d55f69773dc9ed101fa/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:56:06,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa 2023-02-08 08:56:06,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa 2023-02-08 08:56:06,540 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:56:06,541 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for cde43f550d582d55f69773dc9ed101fa 2023-02-08 08:56:06,542 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:56:06,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened cde43f550d582d55f69773dc9ed101fa; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72445733, jitterRate=0.07952554523944855}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:56:06,543 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for cde43f550d582d55f69773dc9ed101fa: Running coprocessor pre-open hook at 1675846566512Writing region info on filesystem at 1675846566512Initializing all the Stores at 1675846566513 (+1 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10h\xCB\x89\xB7f\x9B\x1B\xABuz\xD8\xF5\xD4\xF8Lm"\x10\x173\x13\xF0B\xA0\x0E\x08\x0A\x1F\xA5\x0ExAtd*\x10%\x94\xE4\x99\xB4\xF5\xA5\xB3\x8C\xBC?+\xECC\x86\x162\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846566513Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846566513Cleaning up temporary data from old regions at 1675846566539 (+26 ms)Cleaning up detritus from prior splits at 1675846566539Running coprocessor post-open hooks at 1675846566543 (+4 ms)Region opened successfully at 1675846566543 2023-02-08 08:56:06,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa., pid=121, masterSystemTime=1675846566473 2023-02-08 08:56:06,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:06,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:06,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:06,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e0c4a41d57aa926b6b63e3ff74002885, NAME => 'testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885.', STARTKEY => '2', ENDKEY => ''} 2023-02-08 08:56:06,548 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=cde43f550d582d55f69773dc9ed101fa, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:06,549 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin e0c4a41d57aa926b6b63e3ff74002885 2023-02-08 08:56:06,549 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1675846566548"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846566548"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846566548"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846566548"}]},"ts":"1675846566548"} 2023-02-08 08:56:06,549 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-08 08:56:06,549 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for e0c4a41d57aa926b6b63e3ff74002885 2023-02-08 08:56:06,549 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for e0c4a41d57aa926b6b63e3ff74002885 2023-02-08 08:56:06,551 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=118 2023-02-08 08:56:06,551 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=118, state=SUCCESS; OpenRegionProcedure cde43f550d582d55f69773dc9ed101fa, server=jenkins-hbase4.apache.org,42239,1675846413348 in 228 msec 2023-02-08 08:56:06,552 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=cde43f550d582d55f69773dc9ed101fa, ASSIGN in 385 msec 2023-02-08 08:56:06,580 INFO [StoreOpener-e0c4a41d57aa926b6b63e3ff74002885-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region e0c4a41d57aa926b6b63e3ff74002885 2023-02-08 08:56:06,582 DEBUG [StoreOpener-e0c4a41d57aa926b6b63e3ff74002885-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1 2023-02-08 08:56:06,582 DEBUG [StoreOpener-e0c4a41d57aa926b6b63e3ff74002885-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1 2023-02-08 08:56:06,582 INFO [StoreOpener-e0c4a41d57aa926b6b63e3ff74002885-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e0c4a41d57aa926b6b63e3ff74002885 columnFamilyName family1 2023-02-08 08:56:06,583 INFO [StoreOpener-e0c4a41d57aa926b6b63e3ff74002885-1] regionserver.HStore(310): Store=e0c4a41d57aa926b6b63e3ff74002885/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:56:06,583 INFO [StoreOpener-e0c4a41d57aa926b6b63e3ff74002885-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region e0c4a41d57aa926b6b63e3ff74002885 2023-02-08 08:56:06,584 DEBUG [StoreOpener-e0c4a41d57aa926b6b63e3ff74002885-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2 2023-02-08 08:56:06,585 DEBUG [StoreOpener-e0c4a41d57aa926b6b63e3ff74002885-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2 2023-02-08 08:56:06,585 INFO [StoreOpener-e0c4a41d57aa926b6b63e3ff74002885-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e0c4a41d57aa926b6b63e3ff74002885 columnFamilyName family2 2023-02-08 08:56:06,586 INFO [StoreOpener-e0c4a41d57aa926b6b63e3ff74002885-1] regionserver.HStore(310): Store=e0c4a41d57aa926b6b63e3ff74002885/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-08 08:56:06,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885 2023-02-08 08:56:06,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885 2023-02-08 08:56:06,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-08 08:56:06,590 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for e0c4a41d57aa926b6b63e3ff74002885 2023-02-08 08:56:06,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-08 08:56:06,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened e0c4a41d57aa926b6b63e3ff74002885; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63462627, jitterRate=-0.0543331652879715}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-08 08:56:06,592 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for e0c4a41d57aa926b6b63e3ff74002885: Running coprocessor pre-open hook at 1675846566549Writing region info on filesystem at 1675846566549Initializing all the Stores at 1675846566550 (+1 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10h\xCB\x89\xB7f\x9B\x1B\xABuz\xD8\xF5\xD4\xF8Lm"\x10\x173\x13\xF0B\xA0\x0E\x08\x0A\x1F\xA5\x0ExAtd*\x10%\x94\xE4\x99\xB4\xF5\xA5\xB3\x8C\xBC?+\xECC\x86\x162\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846566550Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1675846566550Cleaning up temporary data from old regions at 1675846566587 (+37 ms)Cleaning up detritus from prior splits at 1675846566588 (+1 ms)Running coprocessor post-open hooks at 1675846566592 (+4 ms)Region opened successfully at 1675846566592 2023-02-08 08:56:06,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885., pid=122, masterSystemTime=1675846566473 2023-02-08 08:56:06,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:06,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:06,594 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=e0c4a41d57aa926b6b63e3ff74002885, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:06,594 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1675846566594"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1675846566594"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1675846566594"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1675846566594"}]},"ts":"1675846566594"} 2023-02-08 08:56:06,596 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=119 2023-02-08 08:56:06,597 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=119, state=SUCCESS; OpenRegionProcedure e0c4a41d57aa926b6b63e3ff74002885, server=jenkins-hbase4.apache.org,42239,1675846413348 in 272 msec 2023-02-08 08:56:06,598 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=119, resume processing ppid=116 2023-02-08 08:56:06,598 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=e0c4a41d57aa926b6b63e3ff74002885, ASSIGN in 431 msec 2023-02-08 08:56:06,598 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-08 08:56:06,599 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1675846566599"}]},"ts":"1675846566599"} 2023-02-08 08:56:06,600 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-02-08 08:56:06,602 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-02-08 08:56:06,603 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 1.2670 sec 2023-02-08 08:56:07,443 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:07,443 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:07,444 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-08 08:56:07,444 INFO [Listener at localhost/40763] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-02-08 08:56:07,450 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:56:07,451 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:56:07,452 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:56:07,453 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:07,453 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:07,454 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:56:07,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-08 08:56:07,455 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-08 08:56:07,455 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:56:07,455 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-08 08:56:07,455 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-08 08:56:07,456 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:07,456 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:07,457 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:56:07,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:56:07,457 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:07,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:56:07,458 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:07,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:07,458 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:56:07,458 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:07,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:56:07,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:07,459 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-08 08:56:07,459 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-08 08:56:07,459 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-08 08:56:07,459 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:56:07,459 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-08 08:56:07,462 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:07,462 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. started... 2023-02-08 08:56:07,463 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cde43f550d582d55f69773dc9ed101fa 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-08 08:56:07,463 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:07,463 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. started... 2023-02-08 08:56:07,463 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 08:56:07,463 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:56:07,464 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing e0c4a41d57aa926b6b63e3ff74002885 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-08 08:56:07,463 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:07,464 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. started... 2023-02-08 08:56:07,465 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing b9a58ce7226f8c74217cc9288a09172f 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-08 08:56:07,518 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302089f015008062147c891e3a10f7cef0118 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302089f015008062147c891e3a10f7cef0118 2023-02-08 08:56:07,524 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020863aa864123db4e9e87f6815e09fbf566 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023020863aa864123db4e9e87f6815e09fbf566 2023-02-08 08:56:07,530 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302089909c6e228d84cc795742784cbac08cb to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302089909c6e228d84cc795742784cbac08cb 2023-02-08 08:56:07,536 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family1/23f09036d3e248b4aef91ca6932833a5 2023-02-08 08:56:07,546 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family1/5d7a5b34c277400983606209b879f46c 2023-02-08 08:56:07,549 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family1/bfec7609ddf6477b9b07b1f79dec38b8 2023-02-08 08:56:07,572 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020856a88e7688bb48b588e77a568e29e63b to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023020856a88e7688bb48b588e77a568e29e63b 2023-02-08 08:56:07,590 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208111145a62b804e49a166e8092bf0faff to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230208111145a62b804e49a166e8092bf0faff 2023-02-08 08:56:07,593 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family2/72db777bbe5e4203833f64767e057f33 2023-02-08 08:56:07,596 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family1/bfec7609ddf6477b9b07b1f79dec38b8 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/bfec7609ddf6477b9b07b1f79dec38b8 2023-02-08 08:56:07,619 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/bfec7609ddf6477b9b07b1f79dec38b8, entries=4, sequenceid=5, filesize=5.5 K 2023-02-08 08:56:07,619 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family2/72db777bbe5e4203833f64767e057f33 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/72db777bbe5e4203833f64767e057f33 2023-02-08 08:56:07,622 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/72db777bbe5e4203833f64767e057f33, entries=2, sequenceid=5, filesize=5.1 K 2023-02-08 08:56:07,623 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for e0c4a41d57aa926b6b63e3ff74002885 in 159ms, sequenceid=5, compaction requested=false 2023-02-08 08:56:07,623 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-02-08 08:56:07,624 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for e0c4a41d57aa926b6b63e3ff74002885: 2023-02-08 08:56:07,624 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:07,624 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 08:56:07,981 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family2/6bbf69aaa7324187833957e2e7b827db 2023-02-08 08:56:07,985 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family1/23f09036d3e248b4aef91ca6932833a5 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/23f09036d3e248b4aef91ca6932833a5 2023-02-08 08:56:07,987 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208f9eb29a7c1fa4150864ef38a896e2389 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230208f9eb29a7c1fa4150864ef38a896e2389 2023-02-08 08:56:08,010 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/23f09036d3e248b4aef91ca6932833a5, entries=4, sequenceid=5, filesize=5.5 K 2023-02-08 08:56:08,011 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family2/6bbf69aaa7324187833957e2e7b827db as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/6bbf69aaa7324187833957e2e7b827db 2023-02-08 08:56:08,014 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/6bbf69aaa7324187833957e2e7b827db, entries=2, sequenceid=5, filesize=5.1 K 2023-02-08 08:56:08,015 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for cde43f550d582d55f69773dc9ed101fa in 552ms, sequenceid=5, compaction requested=false 2023-02-08 08:56:08,015 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cde43f550d582d55f69773dc9ed101fa: 2023-02-08 08:56:08,016 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:08,016 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 08:56:08,392 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family2/03562dd816a94de58cfec4585ec51ac8 2023-02-08 08:56:08,396 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family1/5d7a5b34c277400983606209b879f46c as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/5d7a5b34c277400983606209b879f46c 2023-02-08 08:56:08,418 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/5d7a5b34c277400983606209b879f46c, entries=4, sequenceid=5, filesize=5.5 K 2023-02-08 08:56:08,419 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family2/03562dd816a94de58cfec4585ec51ac8 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/03562dd816a94de58cfec4585ec51ac8 2023-02-08 08:56:08,422 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/03562dd816a94de58cfec4585ec51ac8, entries=2, sequenceid=5, filesize=5.1 K 2023-02-08 08:56:08,422 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for b9a58ce7226f8c74217cc9288a09172f in 957ms, sequenceid=5, compaction requested=false 2023-02-08 08:56:08,422 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for b9a58ce7226f8c74217cc9288a09172f: 2023-02-08 08:56:08,422 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:08,422 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 08:56:08,423 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 08:56:08,423 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:08,423 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-08 08:56:08,423 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-08 08:56:08,430 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:08,430 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:08,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:08,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:08,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:08,430 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:08,430 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:56:08,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:08,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:08,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:08,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:08,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:08,431 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-08 08:56:08,431 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@692025d[Count = 0] remaining members to acquire global barrier 2023-02-08 08:56:08,431 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-08 08:56:08,431 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:08,432 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:08,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:08,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:08,432 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-08 08:56:08,432 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-08 08:56:08,432 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:56:08,432 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:08,433 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:56:08,434 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:08,434 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-08 08:56:08,434 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:56:08,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:08,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:08,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:08,434 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-08 08:56:08,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:08,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:08,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:08,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:08,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:08,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:08,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:08,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:56:08,436 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-08 08:56:08,436 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-08 08:56:08,436 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:56:08,436 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:56:08,436 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-08 08:56:08,436 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:56:08,437 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:08,437 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:08,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:08,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:08,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:08,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:08,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:08,437 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:56:08,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:56:08,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:56:08,438 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:08,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:08,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:08,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:08,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:08,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:08,439 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:08,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:08,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:08,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:08,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:08,441 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:56:08,441 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:08,441 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:56:08,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:56:08,441 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-08 08:56:08,441 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:56:08,441 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:56:08,441 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:08,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:56:08,442 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:08,442 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:08,442 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-08 08:56:08,442 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:56:08,442 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:08,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:56:08,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:56:08,442 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:08,442 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:08,542 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:56:08,542 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:08,542 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:08,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:56:08,544 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:56:08,544 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:56:08,544 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:56:08,546 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:08,546 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:08,546 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:56:08,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-08 08:56:08,548 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-08 08:56:08,548 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-08 08:56:08,548 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:56:08,548 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-08 08:56:08,548 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-08 08:56:08,548 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:08,549 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:08,550 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:08,550 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:56:08,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:56:08,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:56:08,550 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:08,550 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:56:08,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:08,550 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:08,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:56:08,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:08,551 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-08 08:56:08,551 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-08 08:56:08,551 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-08 08:56:08,551 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-08 08:56:08,551 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:56:08,551 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-08 08:56:08,551 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:08,551 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. started... 2023-02-08 08:56:08,551 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cde43f550d582d55f69773dc9ed101fa 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-08 08:56:08,552 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:08,552 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 08:56:08,552 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:56:08,552 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:08,552 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. started... 2023-02-08 08:56:08,552 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. started... 2023-02-08 08:56:08,552 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing e0c4a41d57aa926b6b63e3ff74002885 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-08 08:56:08,552 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing b9a58ce7226f8c74217cc9288a09172f 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-08 08:56:08,601 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208c04ca662d3104207870612ea0c0f48b4 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230208c04ca662d3104207870612ea0c0f48b4 2023-02-08 08:56:08,996 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208a043acd67a2248b1982911d2c3e85a1a to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230208a043acd67a2248b1982911d2c3e85a1a 2023-02-08 08:56:08,999 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208560d40b9130042959a31c153457b0b6d to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230208560d40b9130042959a31c153457b0b6d 2023-02-08 08:56:09,016 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family1/a45bb9edc56f4b1aa954c2c42d01fb45 2023-02-08 08:56:09,412 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family1/e7ba34fc990c44f1b8efff66407925fd 2023-02-08 08:56:09,419 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family1/41917fdf44df42aebad16e108d0c84da 2023-02-08 08:56:09,448 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208c7eca4903a7b417bb801f5bf546a4761 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230208c7eca4903a7b417bb801f5bf546a4761 2023-02-08 08:56:09,451 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302089eb354c6de314f2dabe746562eea56ee to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302089eb354c6de314f2dabe746562eea56ee 2023-02-08 08:56:09,452 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family2/e3ddb8e38f2944feb3556c284eb86154 2023-02-08 08:56:09,455 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family1/a45bb9edc56f4b1aa954c2c42d01fb45 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/a45bb9edc56f4b1aa954c2c42d01fb45 2023-02-08 08:56:09,465 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family2/726930bbcad5432cb6e7ce88c5f20a90 2023-02-08 08:56:09,467 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020850d2889277b34a7aaa41eb51c4a8b5ed to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023020850d2889277b34a7aaa41eb51c4a8b5ed 2023-02-08 08:56:09,469 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family1/e7ba34fc990c44f1b8efff66407925fd as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/e7ba34fc990c44f1b8efff66407925fd 2023-02-08 08:56:09,478 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family2/3337299f8f9440cab974d834cfeda1bc 2023-02-08 08:56:09,481 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family1/41917fdf44df42aebad16e108d0c84da as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/41917fdf44df42aebad16e108d0c84da 2023-02-08 08:56:09,483 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/a45bb9edc56f4b1aa954c2c42d01fb45, entries=4, sequenceid=9, filesize=5.5 K 2023-02-08 08:56:09,484 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family2/e3ddb8e38f2944feb3556c284eb86154 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/e3ddb8e38f2944feb3556c284eb86154 2023-02-08 08:56:09,487 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/e3ddb8e38f2944feb3556c284eb86154, entries=2, sequenceid=9, filesize=5.1 K 2023-02-08 08:56:09,487 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for e0c4a41d57aa926b6b63e3ff74002885 in 935ms, sequenceid=9, compaction requested=false 2023-02-08 08:56:09,488 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for e0c4a41d57aa926b6b63e3ff74002885: 2023-02-08 08:56:09,488 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:09,488 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 08:56:09,494 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/e7ba34fc990c44f1b8efff66407925fd, entries=4, sequenceid=9, filesize=5.5 K 2023-02-08 08:56:09,495 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family2/726930bbcad5432cb6e7ce88c5f20a90 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/726930bbcad5432cb6e7ce88c5f20a90 2023-02-08 08:56:09,498 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/726930bbcad5432cb6e7ce88c5f20a90, entries=2, sequenceid=9, filesize=5.1 K 2023-02-08 08:56:09,498 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for b9a58ce7226f8c74217cc9288a09172f in 946ms, sequenceid=9, compaction requested=false 2023-02-08 08:56:09,498 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for b9a58ce7226f8c74217cc9288a09172f: 2023-02-08 08:56:09,498 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:09,498 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 08:56:09,514 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/41917fdf44df42aebad16e108d0c84da, entries=4, sequenceid=9, filesize=5.5 K 2023-02-08 08:56:09,515 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family2/3337299f8f9440cab974d834cfeda1bc as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/3337299f8f9440cab974d834cfeda1bc 2023-02-08 08:56:09,518 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/3337299f8f9440cab974d834cfeda1bc, entries=2, sequenceid=9, filesize=5.1 K 2023-02-08 08:56:09,519 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for cde43f550d582d55f69773dc9ed101fa in 968ms, sequenceid=9, compaction requested=false 2023-02-08 08:56:09,519 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cde43f550d582d55f69773dc9ed101fa: 2023-02-08 08:56:09,519 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:09,519 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 08:56:09,519 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 08:56:09,519 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:09,519 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-08 08:56:09,519 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-08 08:56:09,520 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:09,520 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:09,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:09,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:09,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:09,521 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:09,521 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:56:09,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:09,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:09,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:09,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:09,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:09,522 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-08 08:56:09,522 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@20f83c77[Count = 0] remaining members to acquire global barrier 2023-02-08 08:56:09,522 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-08 08:56:09,522 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:09,523 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:09,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:09,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:09,523 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-08 08:56:09,523 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-08 08:56:09,523 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:56:09,523 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:09,523 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:56:09,525 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:09,525 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-08 08:56:09,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:09,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:09,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:09,525 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:56:09,525 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-08 08:56:09,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:09,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:09,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:09,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:09,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:09,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:09,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:09,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:56:09,527 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-08 08:56:09,527 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-08 08:56:09,527 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:56:09,527 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:56:09,527 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-08 08:56:09,527 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:56:09,528 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:09,528 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:09,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:09,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:09,528 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:56:09,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:09,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:09,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:09,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:56:09,529 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:09,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:09,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:56:09,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:09,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:09,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:09,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:09,530 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:09,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:09,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:09,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:09,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:09,531 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:09,531 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:56:09,531 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:09,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:56:09,531 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:09,531 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:56:09,531 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:56:09,531 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-08 08:56:09,531 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:56:09,531 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:09,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:56:09,532 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-08 08:56:09,532 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:09,532 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-08 08:56:09,532 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:56:09,532 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:09,532 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:09,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:56:09,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:56:09,632 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:56:09,632 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:09,632 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:09,633 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:56:09,634 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:56:09,635 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:56:09,635 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:56:09,636 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:09,637 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:09,637 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:56:09,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-08 08:56:09,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-08 08:56:09,639 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-08 08:56:09,639 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:56:09,639 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-08 08:56:09,639 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-08 08:56:09,639 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:09,640 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:09,641 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:56:09,641 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:09,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:56:09,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:56:09,641 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:09,641 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:56:09,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:09,641 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:09,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:56:09,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:09,642 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-08 08:56:09,642 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-08 08:56:09,642 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-08 08:56:09,642 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-08 08:56:09,642 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:56:09,642 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-08 08:56:09,642 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:09,642 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 08:56:09,643 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:09,643 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. started... 2023-02-08 08:56:09,643 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. started... 2023-02-08 08:56:09,642 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:09,643 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing b9a58ce7226f8c74217cc9288a09172f 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-08 08:56:09,643 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. started... 2023-02-08 08:56:09,643 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cde43f550d582d55f69773dc9ed101fa 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-08 08:56:09,643 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing e0c4a41d57aa926b6b63e3ff74002885 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-08 08:56:09,643 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:56:09,709 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302084deaf858dfa04534bf24d44b785a6c95 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302084deaf858dfa04534bf24d44b785a6c95 2023-02-08 08:56:10,095 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023020891e001a399814bdd972e719b30ea4328 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023020891e001a399814bdd972e719b30ea4328 2023-02-08 08:56:10,104 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020890db32bd69544183b5e47574b6a1d4fb to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023020890db32bd69544183b5e47574b6a1d4fb 2023-02-08 08:56:10,128 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family1/54d2ce44eed44a63a1d2073318fc620b 2023-02-08 08:56:10,510 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family1/2f0b245a59404081ae2738ed7282af9d 2023-02-08 08:56:10,524 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family1/c334d0a97f7f45a384acbc456c49eda5 2023-02-08 08:56:10,542 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302080e31ea7b18614f9787375c2b760d4654 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202302080e31ea7b18614f9787375c2b760d4654 2023-02-08 08:56:10,560 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023020840a6351d72204adba8b883eae3d8fc4d to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e2023020840a6351d72204adba8b883eae3d8fc4d 2023-02-08 08:56:10,946 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family2/560ec74f81414864a91d31b989dce442 2023-02-08 08:56:10,951 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family1/2f0b245a59404081ae2738ed7282af9d as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/2f0b245a59404081ae2738ed7282af9d 2023-02-08 08:56:10,960 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208162529835c7041ec9992ff86a045d4c4 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230208162529835c7041ec9992ff86a045d4c4 2023-02-08 08:56:10,964 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family2/f80b4e1d4aac4d80a527e91f4f03e334 2023-02-08 08:56:10,974 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family2/4d12cc5f535a4364958a467a9fb4df6f 2023-02-08 08:56:10,976 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family1/54d2ce44eed44a63a1d2073318fc620b as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/54d2ce44eed44a63a1d2073318fc620b 2023-02-08 08:56:10,977 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family1/c334d0a97f7f45a384acbc456c49eda5 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/c334d0a97f7f45a384acbc456c49eda5 2023-02-08 08:56:10,980 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/2f0b245a59404081ae2738ed7282af9d, entries=4, sequenceid=13, filesize=5.5 K 2023-02-08 08:56:10,981 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family2/560ec74f81414864a91d31b989dce442 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/560ec74f81414864a91d31b989dce442 2023-02-08 08:56:10,984 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/560ec74f81414864a91d31b989dce442, entries=2, sequenceid=13, filesize=5.1 K 2023-02-08 08:56:10,985 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for e0c4a41d57aa926b6b63e3ff74002885 in 1342ms, sequenceid=13, compaction requested=true 2023-02-08 08:56:10,985 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for e0c4a41d57aa926b6b63e3ff74002885: 2023-02-08 08:56:10,985 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:10,985 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 08:56:10,999 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/54d2ce44eed44a63a1d2073318fc620b, entries=4, sequenceid=13, filesize=5.5 K 2023-02-08 08:56:11,000 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family2/f80b4e1d4aac4d80a527e91f4f03e334 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/f80b4e1d4aac4d80a527e91f4f03e334 2023-02-08 08:56:11,000 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/c334d0a97f7f45a384acbc456c49eda5, entries=4, sequenceid=13, filesize=5.5 K 2023-02-08 08:56:11,001 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family2/4d12cc5f535a4364958a467a9fb4df6f as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/4d12cc5f535a4364958a467a9fb4df6f 2023-02-08 08:56:11,003 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/f80b4e1d4aac4d80a527e91f4f03e334, entries=2, sequenceid=13, filesize=5.1 K 2023-02-08 08:56:11,004 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/4d12cc5f535a4364958a467a9fb4df6f, entries=2, sequenceid=13, filesize=5.1 K 2023-02-08 08:56:11,004 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for b9a58ce7226f8c74217cc9288a09172f in 1361ms, sequenceid=13, compaction requested=true 2023-02-08 08:56:11,004 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for b9a58ce7226f8c74217cc9288a09172f: 2023-02-08 08:56:11,004 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:11,004 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 08:56:11,005 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for cde43f550d582d55f69773dc9ed101fa in 1361ms, sequenceid=13, compaction requested=true 2023-02-08 08:56:11,005 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cde43f550d582d55f69773dc9ed101fa: 2023-02-08 08:56:11,005 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:11,005 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 08:56:11,005 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 08:56:11,005 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:11,005 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-08 08:56:11,005 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-08 08:56:11,007 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:11,007 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:11,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:11,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:11,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:11,008 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:11,008 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:56:11,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:11,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:11,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:11,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:11,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:11,009 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-08 08:56:11,009 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76d63a09[Count = 0] remaining members to acquire global barrier 2023-02-08 08:56:11,009 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-08 08:56:11,009 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:11,010 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:11,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:11,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:11,010 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-08 08:56:11,010 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-08 08:56:11,010 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:11,010 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:56:11,010 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:56:11,011 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-08 08:56:11,011 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:11,011 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:56:11,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:11,011 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-08 08:56:11,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:11,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:11,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:11,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:11,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:11,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:11,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:11,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:11,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:11,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:56:11,013 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-08 08:56:11,013 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-08 08:56:11,013 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:56:11,013 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:56:11,013 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-08 08:56:11,013 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:56:11,015 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:11,015 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:11,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:11,015 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:56:11,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:11,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:11,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:11,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:11,015 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:11,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:56:11,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:56:11,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:11,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:11,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:11,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:11,016 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:11,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:11,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:11,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:11,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:11,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:11,019 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:11,019 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:56:11,019 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:11,019 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:56:11,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:56:11,019 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:56:11,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:56:11,019 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:11,019 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:11,019 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:11,019 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:11,019 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:11,019 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-08 08:56:11,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:56:11,019 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:56:11,019 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-08 08:56:11,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:56:11,020 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-08 08:56:11,020 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:56:11,120 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:56:11,120 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:11,120 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:11,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:56:11,122 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:56:11,122 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:56:11,122 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. with WAL disabled. Data may be lost in the event of a crash. 2023-02-08 08:56:11,123 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:11,124 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:11,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:56:11,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-08 08:56:11,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-08 08:56:11,126 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-08 08:56:11,126 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:56:11,126 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-08 08:56:11,126 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-08 08:56:11,126 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:11,126 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:11,130 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:56:11,130 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:11,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:56:11,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:56:11,130 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:11,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:11,130 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:56:11,131 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:11,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:56:11,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:11,131 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-08 08:56:11,131 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-08 08:56:11,131 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-08 08:56:11,131 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-08 08:56:11,131 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:56:11,131 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-08 08:56:11,131 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:11,132 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. started... 2023-02-08 08:56:11,132 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:11,132 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. started... 2023-02-08 08:56:11,132 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cde43f550d582d55f69773dc9ed101fa 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-08 08:56:11,132 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing e0c4a41d57aa926b6b63e3ff74002885 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-08 08:56:11,132 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 08:56:11,132 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:56:11,132 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:11,132 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. started... 2023-02-08 08:56:11,132 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing b9a58ce7226f8c74217cc9288a09172f 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-08 08:56:11,181 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208b1c4fd61b50843809d025848a27df959 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230208b1c4fd61b50843809d025848a27df959 2023-02-08 08:56:11,190 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302086ec9215a2b764e74b7063c9b890afff6 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302086ec9215a2b764e74b7063c9b890afff6 2023-02-08 08:56:11,194 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208ad58748bdee74dffa67b15bb25a998ea to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230208ad58748bdee74dffa67b15bb25a998ea 2023-02-08 08:56:11,196 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family1/422c51f946754067a9d1f2b4956edc0b 2023-02-08 08:56:11,610 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family1/ce6ee503646641ee8aa06f3d531c2b54 2023-02-08 08:56:11,614 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family1/b7d5f241e2e04da3ad8c403faf8e8fc1 2023-02-08 08:56:11,643 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208baaa6c4c1a2841a0a5bff17d1aa4f4e1 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230208baaa6c4c1a2841a0a5bff17d1aa4f4e1 2023-02-08 08:56:11,644 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208120465120b744ee69844d63262b16660 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230208120465120b744ee69844d63262b16660 2023-02-08 08:56:11,647 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family2/5596456e3502409b960535ee30326fe2 2023-02-08 08:56:11,648 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family2/12d6988264b244919c4d0d2fe0796edb 2023-02-08 08:56:11,651 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family1/422c51f946754067a9d1f2b4956edc0b as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/422c51f946754067a9d1f2b4956edc0b 2023-02-08 08:56:11,652 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family1/b7d5f241e2e04da3ad8c403faf8e8fc1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/b7d5f241e2e04da3ad8c403faf8e8fc1 2023-02-08 08:56:11,687 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 08:56:11,687 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1ef7f9d3 to 127.0.0.1:61594 2023-02-08 08:56:11,687 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:56:11,688 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 08:56:11,688 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4cb4d02d to 127.0.0.1:61594 2023-02-08 08:56:11,688 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:56:11,700 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/b7d5f241e2e04da3ad8c403faf8e8fc1, entries=4, sequenceid=17, filesize=5.5 K 2023-02-08 08:56:11,701 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family2/12d6988264b244919c4d0d2fe0796edb as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/12d6988264b244919c4d0d2fe0796edb 2023-02-08 08:56:11,704 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/422c51f946754067a9d1f2b4956edc0b, entries=4, sequenceid=17, filesize=5.5 K 2023-02-08 08:56:11,705 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family2/5596456e3502409b960535ee30326fe2 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/5596456e3502409b960535ee30326fe2 2023-02-08 08:56:11,706 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/12d6988264b244919c4d0d2fe0796edb, entries=2, sequenceid=17, filesize=5.1 K 2023-02-08 08:56:11,707 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for e0c4a41d57aa926b6b63e3ff74002885 in 575ms, sequenceid=17, compaction requested=true 2023-02-08 08:56:11,707 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for e0c4a41d57aa926b6b63e3ff74002885: 2023-02-08 08:56:11,707 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:11,707 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 08:56:11,708 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/5596456e3502409b960535ee30326fe2, entries=2, sequenceid=17, filesize=5.1 K 2023-02-08 08:56:11,709 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for b9a58ce7226f8c74217cc9288a09172f in 577ms, sequenceid=17, compaction requested=true 2023-02-08 08:56:11,709 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for b9a58ce7226f8c74217cc9288a09172f: 2023-02-08 08:56:11,709 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:11,709 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 08:56:12,026 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-08 08:56:12,044 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302081c605a992bba497aaad9c4ed4c934ddc to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202302081c605a992bba497aaad9c4ed4c934ddc 2023-02-08 08:56:12,452 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family2/6333740844094d0baf0b593c3b4a4fcb 2023-02-08 08:56:12,456 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family1/ce6ee503646641ee8aa06f3d531c2b54 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/ce6ee503646641ee8aa06f3d531c2b54 2023-02-08 08:56:12,479 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/ce6ee503646641ee8aa06f3d531c2b54, entries=4, sequenceid=17, filesize=5.5 K 2023-02-08 08:56:12,480 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family2/6333740844094d0baf0b593c3b4a4fcb as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/6333740844094d0baf0b593c3b4a4fcb 2023-02-08 08:56:12,484 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/6333740844094d0baf0b593c3b4a4fcb, entries=2, sequenceid=17, filesize=5.1 K 2023-02-08 08:56:12,484 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for cde43f550d582d55f69773dc9ed101fa in 1352ms, sequenceid=17, compaction requested=true 2023-02-08 08:56:12,484 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cde43f550d582d55f69773dc9ed101fa: 2023-02-08 08:56:12,485 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:12,485 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 08:56:12,485 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 08:56:12,485 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:12,485 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-08 08:56:12,485 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-08 08:56:12,486 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:12,486 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:12,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:12,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:12,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:12,486 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:12,486 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:56:12,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:12,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:12,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:12,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:12,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:12,488 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-08 08:56:12,488 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@48560e02[Count = 0] remaining members to acquire global barrier 2023-02-08 08:56:12,488 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-08 08:56:12,488 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:12,489 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:12,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:12,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:12,489 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-08 08:56:12,489 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-08 08:56:12,489 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:56:12,489 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:12,489 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:56:12,490 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-08 08:56:12,490 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:12,490 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:56:12,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:12,490 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-08 08:56:12,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:12,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:12,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:12,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:12,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:12,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:12,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:12,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:12,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:12,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:56:12,492 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-08 08:56:12,492 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-08 08:56:12,492 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:56:12,492 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:56:12,492 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-08 08:56:12,492 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:56:12,494 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:12,494 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:12,494 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:56:12,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:12,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:12,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:12,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:12,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:12,495 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:56:12,495 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:12,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:12,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:56:12,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:12,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:12,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:12,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:12,496 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:12,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:12,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:12,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:12,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:12,498 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:12,498 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:56:12,498 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:56:12,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:56:12,498 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:56:12,498 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-08 08:56:12,498 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:56:12,498 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:12,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:56:12,498 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:12,498 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:12,498 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:12,498 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:12,498 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:12,498 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-08 08:56:12,499 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:56:12,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:56:12,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:56:12,599 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:56:12,599 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:12,599 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:12,600 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:56:12,608 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 08:56:12,614 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 08:56:12,621 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 08:56:12,621 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 08:56:12,701 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:12,701 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:12,702 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-08 08:56:12,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-08 08:56:12,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-08 08:56:12,704 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-08 08:56:12,704 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:56:12,704 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-08 08:56:12,704 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-08 08:56:12,704 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:12,704 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:12,705 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:12,705 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:56:12,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:56:12,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:56:12,705 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:12,705 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-08 08:56:12,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:12,706 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:12,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-08 08:56:12,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:12,706 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-08 08:56:12,706 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-08 08:56:12,706 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-08 08:56:12,707 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-08 08:56:12,707 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-08 08:56:12,707 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-08 08:56:12,707 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:12,707 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. started... 2023-02-08 08:56:12,707 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-08 08:56:12,707 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:12,707 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing cde43f550d582d55f69773dc9ed101fa 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-08 08:56:12,707 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. started... 2023-02-08 08:56:12,707 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-08 08:56:12,707 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:12,707 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing b9a58ce7226f8c74217cc9288a09172f 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-08 08:56:12,707 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. started... 2023-02-08 08:56:12,707 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing e0c4a41d57aa926b6b63e3ff74002885 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-08 08:56:12,746 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family1/5a876ebeb17646b0b8b2b4d728ec98cd 2023-02-08 08:56:12,758 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family1/20411bbbba1d43148177416287e7f11e 2023-02-08 08:56:12,769 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5a876ebeb17646b0b8b2b4d728ec98cd 2023-02-08 08:56:12,778 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family2/de8d012c7fdd4be9be9b0fcce338e463 2023-02-08 08:56:12,780 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for de8d012c7fdd4be9be9b0fcce338e463 2023-02-08 08:56:12,781 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family1/5a876ebeb17646b0b8b2b4d728ec98cd as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/5a876ebeb17646b0b8b2b4d728ec98cd 2023-02-08 08:56:12,794 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 20411bbbba1d43148177416287e7f11e 2023-02-08 08:56:12,803 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5a876ebeb17646b0b8b2b4d728ec98cd 2023-02-08 08:56:12,803 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/5a876ebeb17646b0b8b2b4d728ec98cd, entries=3, sequenceid=23, filesize=5.2 K 2023-02-08 08:56:12,804 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family2/de8d012c7fdd4be9be9b0fcce338e463 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/de8d012c7fdd4be9be9b0fcce338e463 2023-02-08 08:56:12,807 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for de8d012c7fdd4be9be9b0fcce338e463 2023-02-08 08:56:12,807 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/de8d012c7fdd4be9be9b0fcce338e463, entries=1, sequenceid=23, filesize=5.0 K 2023-02-08 08:56:12,808 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for e0c4a41d57aa926b6b63e3ff74002885 in 101ms, sequenceid=23, compaction requested=true 2023-02-08 08:56:12,808 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for e0c4a41d57aa926b6b63e3ff74002885: 2023-02-08 08:56:12,808 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:12,808 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-08 08:56:13,205 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family2/905c1e7ba6ec45aabad8f90dd95034a9 2023-02-08 08:56:13,208 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 905c1e7ba6ec45aabad8f90dd95034a9 2023-02-08 08:56:13,209 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family1/20411bbbba1d43148177416287e7f11e as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/20411bbbba1d43148177416287e7f11e 2023-02-08 08:56:13,241 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 20411bbbba1d43148177416287e7f11e 2023-02-08 08:56:13,241 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/20411bbbba1d43148177416287e7f11e, entries=3, sequenceid=23, filesize=5.2 K 2023-02-08 08:56:13,242 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family2/905c1e7ba6ec45aabad8f90dd95034a9 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/905c1e7ba6ec45aabad8f90dd95034a9 2023-02-08 08:56:13,246 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 905c1e7ba6ec45aabad8f90dd95034a9 2023-02-08 08:56:13,246 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/905c1e7ba6ec45aabad8f90dd95034a9, entries=1, sequenceid=23, filesize=5.0 K 2023-02-08 08:56:13,247 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for cde43f550d582d55f69773dc9ed101fa in 540ms, sequenceid=23, compaction requested=true 2023-02-08 08:56:13,247 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for cde43f550d582d55f69773dc9ed101fa: 2023-02-08 08:56:13,247 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:13,247 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-08 08:56:13,555 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family1/4c4cfd8cd12a4980ac816d8a535f93a1 2023-02-08 08:56:13,577 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4c4cfd8cd12a4980ac816d8a535f93a1 2023-02-08 08:56:14,388 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family2/530e3a475fc5468fa3e1c1ede0e0a184 2023-02-08 08:56:14,391 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 530e3a475fc5468fa3e1c1ede0e0a184 2023-02-08 08:56:14,392 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family1/4c4cfd8cd12a4980ac816d8a535f93a1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/4c4cfd8cd12a4980ac816d8a535f93a1 2023-02-08 08:56:14,415 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4c4cfd8cd12a4980ac816d8a535f93a1 2023-02-08 08:56:14,415 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/4c4cfd8cd12a4980ac816d8a535f93a1, entries=3, sequenceid=23, filesize=5.2 K 2023-02-08 08:56:14,415 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family2/530e3a475fc5468fa3e1c1ede0e0a184 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/530e3a475fc5468fa3e1c1ede0e0a184 2023-02-08 08:56:14,419 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 530e3a475fc5468fa3e1c1ede0e0a184 2023-02-08 08:56:14,419 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/530e3a475fc5468fa3e1c1ede0e0a184, entries=1, sequenceid=23, filesize=5.0 K 2023-02-08 08:56:14,419 INFO [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for b9a58ce7226f8c74217cc9288a09172f in 1712ms, sequenceid=23, compaction requested=true 2023-02-08 08:56:14,420 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for b9a58ce7226f8c74217cc9288a09172f: 2023-02-08 08:56:14,420 DEBUG [rs(jenkins-hbase4.apache.org,42239,1675846413348)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:14,420 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-08 08:56:14,420 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-08 08:56:14,420 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:14,420 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-08 08:56:14,420 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-08 08:56:14,421 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:14,421 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:14,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:14,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:14,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:14,422 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:14,422 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-08 08:56:14,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:14,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:14,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:14,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:14,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:14,423 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,42239,1675846413348' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-08 08:56:14,423 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ec07dbf[Count = 0] remaining members to acquire global barrier 2023-02-08 08:56:14,423 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-08 08:56:14,423 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:14,425 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:14,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:14,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:14,425 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-08 08:56:14,425 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-08 08:56:14,425 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:14,425 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-08 08:56:14,425 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,42239,1675846413348' in zk 2023-02-08 08:56:14,426 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-08 08:56:14,426 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:56:14,426 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:14,426 DEBUG [member: 'jenkins-hbase4.apache.org,42239,1675846413348' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-08 08:56:14,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:14,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:14,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:14,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:14,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:14,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:14,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:14,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:14,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:14,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:14,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,42239,1675846413348': 2023-02-08 08:56:14,428 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,42239,1675846413348' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-08 08:56:14,428 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-08 08:56:14,429 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-08 08:56:14,429 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-08 08:56:14,429 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-08 08:56:14,429 INFO [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-08 08:56:14,430 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:14,430 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:14,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:14,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:14,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-08 08:56:14,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-08 08:56:14,430 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:56:14,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:14,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-08 08:56:14,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:56:14,430 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:14,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:56:14,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:14,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:14,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-08 08:56:14,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:14,437 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:14,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:14,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-08 08:56:14,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-08 08:56:14,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:14,439 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:14,439 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-08 08:56:14,439 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:14,439 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-08 08:56:14,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-08 08:56:14,439 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-08 08:56:14,439 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-08 08:56:14,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-08 08:56:14,439 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-08 08:56:14,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-08 08:56:14,439 DEBUG [(jenkins-hbase4.apache.org,37869,1675846412232)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-08 08:56:14,439 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-08 08:56:14,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-08 08:56:14,439 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-08 08:56:14,439 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:14,440 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-08 08:56:14,440 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:14,440 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-08 08:56:14,440 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-08 08:56:14,540 DEBUG [Listener at localhost/40763] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-08 08:56:14,540 INFO [Listener at localhost/40763] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:14,540 INFO [Listener at localhost/40763] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:14,541 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37869] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-08 08:56:14,541 DEBUG [Listener at localhost/40763] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 08:56:14,541 DEBUG [Listener at localhost/40763] regionserver.HStore(1912): cde43f550d582d55f69773dc9ed101fa/family1 is initiating major compaction (all files) 2023-02-08 08:56:14,541 INFO [Listener at localhost/40763] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:56:14,541 INFO [Listener at localhost/40763] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:56:14,542 INFO [Listener at localhost/40763] regionserver.HRegion(2258): Starting compaction of cde43f550d582d55f69773dc9ed101fa/family1 in testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:14,542 INFO [Listener at localhost/40763] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/23f09036d3e248b4aef91ca6932833a5, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/41917fdf44df42aebad16e108d0c84da, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/c334d0a97f7f45a384acbc456c49eda5, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/ce6ee503646641ee8aa06f3d531c2b54, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/20411bbbba1d43148177416287e7f11e] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp, totalSize=27.1 K 2023-02-08 08:56:14,542 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 23f09036d3e248b4aef91ca6932833a5, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846567451 2023-02-08 08:56:14,543 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 41917fdf44df42aebad16e108d0c84da, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846568544 2023-02-08 08:56:14,543 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting c334d0a97f7f45a384acbc456c49eda5, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846569635 2023-02-08 08:56:14,543 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting ce6ee503646641ee8aa06f3d531c2b54, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846571122 2023-02-08 08:56:14,544 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 20411bbbba1d43148177416287e7f11e, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 08:56:14,552 INFO [Listener at localhost/40763] throttle.PressureAwareThroughputController(145): cde43f550d582d55f69773dc9ed101fa#family1#compaction#197 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:56:14,588 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230208d478052e31ac4370a516048eb7c5f0c4_del 2023-02-08 08:56:14,588 INFO [Listener at localhost/40763] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208d478052e31ac4370a516048eb7c5f0c4_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230208d478052e31ac4370a516048eb7c5f0c4_del 2023-02-08 08:56:15,039 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 776eec2e6ff64dccb644140bf3503909 2023-02-08 08:56:15,039 DEBUG [Listener at localhost/40763] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family1/776eec2e6ff64dccb644140bf3503909 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/776eec2e6ff64dccb644140bf3503909 2023-02-08 08:56:15,068 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 776eec2e6ff64dccb644140bf3503909 2023-02-08 08:56:15,069 INFO [Listener at localhost/40763] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in cde43f550d582d55f69773dc9ed101fa/family1 of cde43f550d582d55f69773dc9ed101fa into 776eec2e6ff64dccb644140bf3503909(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:56:15,069 DEBUG [Listener at localhost/40763] regionserver.HRegion(2288): Compaction status journal for cde43f550d582d55f69773dc9ed101fa: 2023-02-08 08:56:15,069 DEBUG [Listener at localhost/40763] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 08:56:15,070 DEBUG [Listener at localhost/40763] regionserver.HStore(1912): cde43f550d582d55f69773dc9ed101fa/family2 is initiating major compaction (all files) 2023-02-08 08:56:15,070 INFO [Listener at localhost/40763] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:56:15,070 INFO [Listener at localhost/40763] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:56:15,072 INFO [Listener at localhost/40763] regionserver.HRegion(2258): Starting compaction of cde43f550d582d55f69773dc9ed101fa/family2 in testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:15,072 INFO [Listener at localhost/40763] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/6bbf69aaa7324187833957e2e7b827db, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/3337299f8f9440cab974d834cfeda1bc, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/4d12cc5f535a4364958a467a9fb4df6f, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/6333740844094d0baf0b593c3b4a4fcb, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/905c1e7ba6ec45aabad8f90dd95034a9] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp, totalSize=25.4 K 2023-02-08 08:56:15,072 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 6bbf69aaa7324187833957e2e7b827db, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846567451 2023-02-08 08:56:15,073 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 3337299f8f9440cab974d834cfeda1bc, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846568544 2023-02-08 08:56:15,073 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 4d12cc5f535a4364958a467a9fb4df6f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846569635 2023-02-08 08:56:15,074 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 6333740844094d0baf0b593c3b4a4fcb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846571122 2023-02-08 08:56:15,074 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 905c1e7ba6ec45aabad8f90dd95034a9, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 08:56:15,089 INFO [Listener at localhost/40763] throttle.PressureAwareThroughputController(145): cde43f550d582d55f69773dc9ed101fa#family2#compaction#198 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:56:15,113 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/23f09036d3e248b4aef91ca6932833a5, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/41917fdf44df42aebad16e108d0c84da, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/c334d0a97f7f45a384acbc456c49eda5, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/ce6ee503646641ee8aa06f3d531c2b54, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/20411bbbba1d43148177416287e7f11e] to archive 2023-02-08 08:56:15,114 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:56:15,116 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/23f09036d3e248b4aef91ca6932833a5 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/23f09036d3e248b4aef91ca6932833a5 2023-02-08 08:56:15,117 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/41917fdf44df42aebad16e108d0c84da to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/41917fdf44df42aebad16e108d0c84da 2023-02-08 08:56:15,118 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/c334d0a97f7f45a384acbc456c49eda5 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/c334d0a97f7f45a384acbc456c49eda5 2023-02-08 08:56:15,119 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/ce6ee503646641ee8aa06f3d531c2b54 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/ce6ee503646641ee8aa06f3d531c2b54 2023-02-08 08:56:15,120 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/20411bbbba1d43148177416287e7f11e to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/20411bbbba1d43148177416287e7f11e 2023-02-08 08:56:15,506 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230208b734bd6397fb431f8d68e233154d6d99_del 2023-02-08 08:56:15,506 INFO [Listener at localhost/40763] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230208b734bd6397fb431f8d68e233154d6d99_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230208b734bd6397fb431f8d68e233154d6d99_del 2023-02-08 08:56:15,517 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2364bf91e691401eaf9208a3c8ce0ac5 2023-02-08 08:56:15,518 DEBUG [Listener at localhost/40763] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/.tmp/family2/2364bf91e691401eaf9208a3c8ce0ac5 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/2364bf91e691401eaf9208a3c8ce0ac5 2023-02-08 08:56:15,521 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2364bf91e691401eaf9208a3c8ce0ac5 2023-02-08 08:56:15,522 INFO [Listener at localhost/40763] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in cde43f550d582d55f69773dc9ed101fa/family2 of cde43f550d582d55f69773dc9ed101fa into 2364bf91e691401eaf9208a3c8ce0ac5(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:56:15,522 DEBUG [Listener at localhost/40763] regionserver.HRegion(2288): Compaction status journal for cde43f550d582d55f69773dc9ed101fa: 2023-02-08 08:56:15,522 DEBUG [Listener at localhost/40763] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 08:56:15,522 DEBUG [Listener at localhost/40763] regionserver.HStore(1912): e0c4a41d57aa926b6b63e3ff74002885/family1 is initiating major compaction (all files) 2023-02-08 08:56:15,523 INFO [Listener at localhost/40763] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:56:15,523 INFO [Listener at localhost/40763] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:56:15,523 INFO [Listener at localhost/40763] regionserver.HRegion(2258): Starting compaction of e0c4a41d57aa926b6b63e3ff74002885/family1 in testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:15,523 INFO [Listener at localhost/40763] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/bfec7609ddf6477b9b07b1f79dec38b8, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/a45bb9edc56f4b1aa954c2c42d01fb45, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/2f0b245a59404081ae2738ed7282af9d, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/b7d5f241e2e04da3ad8c403faf8e8fc1, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/5a876ebeb17646b0b8b2b4d728ec98cd] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp, totalSize=27.1 K 2023-02-08 08:56:15,523 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting bfec7609ddf6477b9b07b1f79dec38b8, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846567451 2023-02-08 08:56:15,524 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting a45bb9edc56f4b1aa954c2c42d01fb45, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846568544 2023-02-08 08:56:15,524 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 2f0b245a59404081ae2738ed7282af9d, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846569635 2023-02-08 08:56:15,524 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting b7d5f241e2e04da3ad8c403faf8e8fc1, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846571122 2023-02-08 08:56:15,524 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 5a876ebeb17646b0b8b2b4d728ec98cd, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 08:56:15,531 INFO [Listener at localhost/40763] throttle.PressureAwareThroughputController(145): e0c4a41d57aa926b6b63e3ff74002885#family1#compaction#199 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:56:15,613 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/6bbf69aaa7324187833957e2e7b827db, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/3337299f8f9440cab974d834cfeda1bc, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/4d12cc5f535a4364958a467a9fb4df6f, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/6333740844094d0baf0b593c3b4a4fcb, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/905c1e7ba6ec45aabad8f90dd95034a9] to archive 2023-02-08 08:56:15,614 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:56:15,615 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/6bbf69aaa7324187833957e2e7b827db to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/6bbf69aaa7324187833957e2e7b827db 2023-02-08 08:56:15,616 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/3337299f8f9440cab974d834cfeda1bc to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/3337299f8f9440cab974d834cfeda1bc 2023-02-08 08:56:15,617 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/4d12cc5f535a4364958a467a9fb4df6f to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/4d12cc5f535a4364958a467a9fb4df6f 2023-02-08 08:56:15,618 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/6333740844094d0baf0b593c3b4a4fcb to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/6333740844094d0baf0b593c3b4a4fcb 2023-02-08 08:56:15,618 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/905c1e7ba6ec45aabad8f90dd95034a9 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family2/905c1e7ba6ec45aabad8f90dd95034a9 2023-02-08 08:56:15,968 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302084f4fc8360d8e428dafd1f636514bc21b_del 2023-02-08 08:56:15,969 INFO [Listener at localhost/40763] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302084f4fc8360d8e428dafd1f636514bc21b_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302084f4fc8360d8e428dafd1f636514bc21b_del 2023-02-08 08:56:16,824 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 61973e06b122442a89342603729585fd 2023-02-08 08:56:16,825 DEBUG [Listener at localhost/40763] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family1/61973e06b122442a89342603729585fd as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/61973e06b122442a89342603729585fd 2023-02-08 08:56:16,846 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 61973e06b122442a89342603729585fd 2023-02-08 08:56:16,847 INFO [Listener at localhost/40763] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in e0c4a41d57aa926b6b63e3ff74002885/family1 of e0c4a41d57aa926b6b63e3ff74002885 into 61973e06b122442a89342603729585fd(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 08:56:16,847 DEBUG [Listener at localhost/40763] regionserver.HRegion(2288): Compaction status journal for e0c4a41d57aa926b6b63e3ff74002885: 2023-02-08 08:56:16,847 DEBUG [Listener at localhost/40763] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 08:56:16,847 DEBUG [Listener at localhost/40763] regionserver.HStore(1912): e0c4a41d57aa926b6b63e3ff74002885/family2 is initiating major compaction (all files) 2023-02-08 08:56:16,847 INFO [Listener at localhost/40763] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:56:16,847 INFO [Listener at localhost/40763] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:56:16,847 INFO [Listener at localhost/40763] regionserver.HRegion(2258): Starting compaction of e0c4a41d57aa926b6b63e3ff74002885/family2 in testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:16,848 INFO [Listener at localhost/40763] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/72db777bbe5e4203833f64767e057f33, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/e3ddb8e38f2944feb3556c284eb86154, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/560ec74f81414864a91d31b989dce442, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/12d6988264b244919c4d0d2fe0796edb, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/de8d012c7fdd4be9be9b0fcce338e463] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp, totalSize=25.4 K 2023-02-08 08:56:16,848 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 72db777bbe5e4203833f64767e057f33, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846567451 2023-02-08 08:56:16,848 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting e3ddb8e38f2944feb3556c284eb86154, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846568544 2023-02-08 08:56:16,849 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 560ec74f81414864a91d31b989dce442, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846569635 2023-02-08 08:56:16,849 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 12d6988264b244919c4d0d2fe0796edb, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846571122 2023-02-08 08:56:16,849 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting de8d012c7fdd4be9be9b0fcce338e463, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 08:56:16,859 INFO [Listener at localhost/40763] throttle.PressureAwareThroughputController(145): e0c4a41d57aa926b6b63e3ff74002885#family2#compaction#200 average throughput is 0.26 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:56:16,913 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/bfec7609ddf6477b9b07b1f79dec38b8, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/a45bb9edc56f4b1aa954c2c42d01fb45, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/2f0b245a59404081ae2738ed7282af9d, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/b7d5f241e2e04da3ad8c403faf8e8fc1, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/5a876ebeb17646b0b8b2b4d728ec98cd] to archive 2023-02-08 08:56:16,913 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:56:16,915 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/bfec7609ddf6477b9b07b1f79dec38b8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/bfec7609ddf6477b9b07b1f79dec38b8 2023-02-08 08:56:16,916 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/a45bb9edc56f4b1aa954c2c42d01fb45 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/a45bb9edc56f4b1aa954c2c42d01fb45 2023-02-08 08:56:16,917 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/2f0b245a59404081ae2738ed7282af9d to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/2f0b245a59404081ae2738ed7282af9d 2023-02-08 08:56:16,918 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/b7d5f241e2e04da3ad8c403faf8e8fc1 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/b7d5f241e2e04da3ad8c403faf8e8fc1 2023-02-08 08:56:16,919 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/5a876ebeb17646b0b8b2b4d728ec98cd to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/5a876ebeb17646b0b8b2b4d728ec98cd 2023-02-08 08:56:17,269 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230208284b8be3fae54a33ab5b68a3b605bb0b_del 2023-02-08 08:56:17,270 INFO [Listener at localhost/40763] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208284b8be3fae54a33ab5b68a3b605bb0b_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230208284b8be3fae54a33ab5b68a3b605bb0b_del 2023-02-08 08:56:17,280 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9d2aa51d32234eb6a57e78157db437aa 2023-02-08 08:56:17,281 DEBUG [Listener at localhost/40763] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/.tmp/family2/9d2aa51d32234eb6a57e78157db437aa as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/9d2aa51d32234eb6a57e78157db437aa 2023-02-08 08:56:17,284 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9d2aa51d32234eb6a57e78157db437aa 2023-02-08 08:56:17,285 INFO [Listener at localhost/40763] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in e0c4a41d57aa926b6b63e3ff74002885/family2 of e0c4a41d57aa926b6b63e3ff74002885 into 9d2aa51d32234eb6a57e78157db437aa(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:56:17,285 DEBUG [Listener at localhost/40763] regionserver.HRegion(2288): Compaction status journal for e0c4a41d57aa926b6b63e3ff74002885: 2023-02-08 08:56:17,285 DEBUG [Listener at localhost/40763] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 08:56:17,285 DEBUG [Listener at localhost/40763] regionserver.HStore(1912): b9a58ce7226f8c74217cc9288a09172f/family1 is initiating major compaction (all files) 2023-02-08 08:56:17,285 INFO [Listener at localhost/40763] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:56:17,285 INFO [Listener at localhost/40763] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:56:17,285 INFO [Listener at localhost/40763] regionserver.HRegion(2258): Starting compaction of b9a58ce7226f8c74217cc9288a09172f/family1 in testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:17,285 INFO [Listener at localhost/40763] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/5d7a5b34c277400983606209b879f46c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/e7ba34fc990c44f1b8efff66407925fd, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/54d2ce44eed44a63a1d2073318fc620b, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/422c51f946754067a9d1f2b4956edc0b, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/4c4cfd8cd12a4980ac816d8a535f93a1] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp, totalSize=27.1 K 2023-02-08 08:56:17,286 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 5d7a5b34c277400983606209b879f46c, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846567450 2023-02-08 08:56:17,286 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting e7ba34fc990c44f1b8efff66407925fd, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846568544 2023-02-08 08:56:17,286 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 54d2ce44eed44a63a1d2073318fc620b, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846569634 2023-02-08 08:56:17,287 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 422c51f946754067a9d1f2b4956edc0b, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846571122 2023-02-08 08:56:17,287 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 4c4cfd8cd12a4980ac816d8a535f93a1, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 08:56:17,294 INFO [Listener at localhost/40763] throttle.PressureAwareThroughputController(145): b9a58ce7226f8c74217cc9288a09172f#family1#compaction#201 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:56:17,315 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/72db777bbe5e4203833f64767e057f33, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/e3ddb8e38f2944feb3556c284eb86154, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/560ec74f81414864a91d31b989dce442, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/12d6988264b244919c4d0d2fe0796edb, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/de8d012c7fdd4be9be9b0fcce338e463] to archive 2023-02-08 08:56:17,316 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:56:17,317 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/72db777bbe5e4203833f64767e057f33 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/72db777bbe5e4203833f64767e057f33 2023-02-08 08:56:17,318 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/e3ddb8e38f2944feb3556c284eb86154 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/e3ddb8e38f2944feb3556c284eb86154 2023-02-08 08:56:17,319 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/560ec74f81414864a91d31b989dce442 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/560ec74f81414864a91d31b989dce442 2023-02-08 08:56:17,320 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/12d6988264b244919c4d0d2fe0796edb to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/12d6988264b244919c4d0d2fe0796edb 2023-02-08 08:56:17,322 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/de8d012c7fdd4be9be9b0fcce338e463 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family2/de8d012c7fdd4be9be9b0fcce338e463 2023-02-08 08:56:17,331 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302082476c309d3584d9f8af8198d02ad5eda_del 2023-02-08 08:56:17,331 INFO [Listener at localhost/40763] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302082476c309d3584d9f8af8198d02ad5eda_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302082476c309d3584d9f8af8198d02ad5eda_del 2023-02-08 08:56:18,184 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6d012c967380497bb2572d054fb3fd68 2023-02-08 08:56:18,185 DEBUG [Listener at localhost/40763] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family1/6d012c967380497bb2572d054fb3fd68 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/6d012c967380497bb2572d054fb3fd68 2023-02-08 08:56:18,207 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6d012c967380497bb2572d054fb3fd68 2023-02-08 08:56:18,208 INFO [Listener at localhost/40763] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in b9a58ce7226f8c74217cc9288a09172f/family1 of b9a58ce7226f8c74217cc9288a09172f into 6d012c967380497bb2572d054fb3fd68(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-08 08:56:18,208 DEBUG [Listener at localhost/40763] regionserver.HRegion(2288): Compaction status journal for b9a58ce7226f8c74217cc9288a09172f: 2023-02-08 08:56:18,208 DEBUG [Listener at localhost/40763] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-08 08:56:18,208 DEBUG [Listener at localhost/40763] regionserver.HStore(1912): b9a58ce7226f8c74217cc9288a09172f/family2 is initiating major compaction (all files) 2023-02-08 08:56:18,208 INFO [Listener at localhost/40763] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-08 08:56:18,208 INFO [Listener at localhost/40763] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-08 08:56:18,209 INFO [Listener at localhost/40763] regionserver.HRegion(2258): Starting compaction of b9a58ce7226f8c74217cc9288a09172f/family2 in testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:18,209 INFO [Listener at localhost/40763] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/03562dd816a94de58cfec4585ec51ac8, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/726930bbcad5432cb6e7ce88c5f20a90, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/f80b4e1d4aac4d80a527e91f4f03e334, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/5596456e3502409b960535ee30326fe2, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/530e3a475fc5468fa3e1c1ede0e0a184] into tmpdir=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp, totalSize=25.4 K 2023-02-08 08:56:18,209 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 03562dd816a94de58cfec4585ec51ac8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1675846567450 2023-02-08 08:56:18,210 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 726930bbcad5432cb6e7ce88c5f20a90, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1675846568544 2023-02-08 08:56:18,210 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting f80b4e1d4aac4d80a527e91f4f03e334, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1675846569634 2023-02-08 08:56:18,210 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 5596456e3502409b960535ee30326fe2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1675846571122 2023-02-08 08:56:18,211 DEBUG [Listener at localhost/40763] compactions.Compactor(207): Compacting 530e3a475fc5468fa3e1c1ede0e0a184, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-08 08:56:18,213 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/5d7a5b34c277400983606209b879f46c, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/e7ba34fc990c44f1b8efff66407925fd, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/54d2ce44eed44a63a1d2073318fc620b, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/422c51f946754067a9d1f2b4956edc0b, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/4c4cfd8cd12a4980ac816d8a535f93a1] to archive 2023-02-08 08:56:18,214 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:56:18,215 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/5d7a5b34c277400983606209b879f46c to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/5d7a5b34c277400983606209b879f46c 2023-02-08 08:56:18,216 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/e7ba34fc990c44f1b8efff66407925fd to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/e7ba34fc990c44f1b8efff66407925fd 2023-02-08 08:56:18,217 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/54d2ce44eed44a63a1d2073318fc620b to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/54d2ce44eed44a63a1d2073318fc620b 2023-02-08 08:56:18,219 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/422c51f946754067a9d1f2b4956edc0b to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/422c51f946754067a9d1f2b4956edc0b 2023-02-08 08:56:18,219 INFO [Listener at localhost/40763] throttle.PressureAwareThroughputController(145): b9a58ce7226f8c74217cc9288a09172f#family2#compaction#202 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-08 08:56:18,220 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/4c4cfd8cd12a4980ac816d8a535f93a1 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/4c4cfd8cd12a4980ac816d8a535f93a1 2023-02-08 08:56:18,626 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230208445c18fc65a249f68b15d9b4d84dd00c_del 2023-02-08 08:56:18,626 INFO [Listener at localhost/40763] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208445c18fc65a249f68b15d9b4d84dd00c_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230208445c18fc65a249f68b15d9b4d84dd00c_del 2023-02-08 08:56:19,439 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c7f25a5186d54926805a398f82f63723 2023-02-08 08:56:19,439 DEBUG [Listener at localhost/40763] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/.tmp/family2/c7f25a5186d54926805a398f82f63723 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/c7f25a5186d54926805a398f82f63723 2023-02-08 08:56:19,442 INFO [Listener at localhost/40763] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c7f25a5186d54926805a398f82f63723 2023-02-08 08:56:19,443 INFO [Listener at localhost/40763] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in b9a58ce7226f8c74217cc9288a09172f/family2 of b9a58ce7226f8c74217cc9288a09172f into c7f25a5186d54926805a398f82f63723(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-08 08:56:19,443 DEBUG [Listener at localhost/40763] regionserver.HRegion(2288): Compaction status journal for b9a58ce7226f8c74217cc9288a09172f: 2023-02-08 08:56:19,447 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 08:56:19,449 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 08:56:19,453 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 08:56:19,454 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 08:56:19,458 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37869] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10h\xCB\x89\xB7f\x9B\x1B\xABuz\xD8\xF5\xD4\xF8Lm"\x10\x173\x13\xF0B\xA0\x0E\x08\x0A\x1F\xA5\x0ExAtd*\x10%\x94\xE4\x99\xB4\xF5\xA5\xB3\x8C\xBC?+\xECC\x86\x162\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-02-08 08:56:19,484 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-08 08:56:19,485 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-08 08:56:19,485 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:56:19,485 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1675847179485 2023-02-08 08:56:19,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-08 08:56:19,521 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/03562dd816a94de58cfec4585ec51ac8, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/726930bbcad5432cb6e7ce88c5f20a90, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/f80b4e1d4aac4d80a527e91f4f03e334, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/5596456e3502409b960535ee30326fe2, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/530e3a475fc5468fa3e1c1ede0e0a184] to archive 2023-02-08 08:56:19,522 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:56:19,523 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/03562dd816a94de58cfec4585ec51ac8 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/03562dd816a94de58cfec4585ec51ac8 2023-02-08 08:56:19,525 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/726930bbcad5432cb6e7ce88c5f20a90 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/726930bbcad5432cb6e7ce88c5f20a90 2023-02-08 08:56:19,526 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/f80b4e1d4aac4d80a527e91f4f03e334 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/f80b4e1d4aac4d80a527e91f4f03e334 2023-02-08 08:56:19,526 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/5596456e3502409b960535ee30326fe2 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/5596456e3502409b960535ee30326fe2 2023-02-08 08:56:19,527 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/530e3a475fc5468fa3e1c1ede0e0a184 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family2/530e3a475fc5468fa3e1c1ede0e0a184 2023-02-08 08:56:19,791 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-02-08 08:56:19,821 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302082476c309d3584d9f8af8198d02ad5eda_del 2023-02-08 08:56:19,842 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230208d478052e31ac4370a516048eb7c5f0c4_del 2023-02-08 08:56:19,863 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302084f4fc8360d8e428dafd1f636514bc21b_del 2023-02-08 08:56:19,863 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-02-08 08:56:19,864 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] zookeeper.ReadOnlyZKClient(139): Connect 0x6c01db0f to 127.0.0.1:61594 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-08 08:56:19,869 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@646f44b3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-08 08:56:19,870 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230208 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-08 08:56:20,842 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230208a48d00604a1e4667bacab578dbad1d97 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230208a48d00604a1e4667bacab578dbad1d97 2023-02-08 08:56:20,843 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230208 table='testMajorCompactionFromAdmin' column='family1' 2023-02-08 08:56:20,854 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-08 08:56:20,857 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42236, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-08 08:56:20,868 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:20,868 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:20,869 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-08 08:56:20,871 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54024, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-08 08:56:20,898 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230208/family1/4932a234a5104e5b9b9f72a49c049a35 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-02-08 08:56:20,899 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230208/family1/4932a234a5104e5b9b9f72a49c049a35}] 2023-02-08 08:56:20,903 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230208/family1/4932a234a5104e5b9b9f72a49c049a35 for inclusion in b9a58ce7226f8c74217cc9288a09172f/family1 2023-02-08 08:56:20,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-02-08 08:56:20,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(777): Region bounds: first= last=1 2023-02-08 08:56:20,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for b9a58ce7226f8c74217cc9288a09172f: 2023-02-08 08:56:20,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230208/family1/4932a234a5104e5b9b9f72a49c049a35 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionFromAdmin__ak02s4d64gk4oqjif9tr30cd4sbfqbcr5ti7rkbb11mrqgsagfi17uv7d087dkeu/family1/4932a234a5104e5b9b9f72a49c049a35 2023-02-08 08:56:20,937 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionFromAdmin__ak02s4d64gk4oqjif9tr30cd4sbfqbcr5ti7rkbb11mrqgsagfi17uv7d087dkeu/family1/4932a234a5104e5b9b9f72a49c049a35 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/22f61816d2734a6591d6305bb316f607_SeqId_28_ 2023-02-08 08:56:20,937 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionFromAdmin__ak02s4d64gk4oqjif9tr30cd4sbfqbcr5ti7rkbb11mrqgsagfi17uv7d087dkeu/family1/4932a234a5104e5b9b9f72a49c049a35 into b9a58ce7226f8c74217cc9288a09172f/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/22f61816d2734a6591d6305bb316f607_SeqId_28_ - updating store file list. 2023-02-08 08:56:20,958 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/22f61816d2734a6591d6305bb316f607_SeqId_28_ into b9a58ce7226f8c74217cc9288a09172f/family1 2023-02-08 08:56:20,959 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionFromAdmin__ak02s4d64gk4oqjif9tr30cd4sbfqbcr5ti7rkbb11mrqgsagfi17uv7d087dkeu/family1/4932a234a5104e5b9b9f72a49c049a35 into b9a58ce7226f8c74217cc9288a09172f/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/family1/22f61816d2734a6591d6305bb316f607_SeqId_28_) 2023-02-08 08:56:20,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionFromAdmin__ak02s4d64gk4oqjif9tr30cd4sbfqbcr5ti7rkbb11mrqgsagfi17uv7d087dkeu/family1/4932a234a5104e5b9b9f72a49c049a35 2023-02-08 08:56:20,961 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230208 table='testMajorCompactionFromAdmin' column='family1' 2023-02-08 08:56:20,962 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:56:20,963 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302084deaf858dfa04534bf24d44b785a6c95 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302084deaf858dfa04534bf24d44b785a6c95 2023-02-08 08:56:20,964 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302089909c6e228d84cc795742784cbac08cb to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302089909c6e228d84cc795742784cbac08cb 2023-02-08 08:56:20,965 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230208a043acd67a2248b1982911d2c3e85a1a to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230208a043acd67a2248b1982911d2c3e85a1a 2023-02-08 08:56:20,966 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230208b1c4fd61b50843809d025848a27df959 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230208b1c4fd61b50843809d025848a27df959 2023-02-08 08:56:20,966 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-08 08:56:20,966 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230208' 2023-02-08 08:56:20,966 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230208 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-08 08:56:20,966 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230208 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-08 08:56:21,917 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023020887ef3a40ab3f4bc9a1f528aa34b702fc to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023020887ef3a40ab3f4bc9a1f528aa34b702fc 2023-02-08 08:56:21,918 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230208 table='testMajorCompactionFromAdmin' column='family1' 2023-02-08 08:56:21,933 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:21,933 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:21,959 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230208/family1/2b6de9d290d84335b95f4f90c9cf4e24 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-02-08 08:56:21,960 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230208/family1/2b6de9d290d84335b95f4f90c9cf4e24}] 2023-02-08 08:56:21,962 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230208/family1/2b6de9d290d84335b95f4f90c9cf4e24 for inclusion in cde43f550d582d55f69773dc9ed101fa/family1 2023-02-08 08:56:21,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-02-08 08:56:21,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-08 08:56:21,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for cde43f550d582d55f69773dc9ed101fa: 2023-02-08 08:56:21,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230208/family1/2b6de9d290d84335b95f4f90c9cf4e24 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionFromAdmin__6t191ppsonmae4j5lvocfp0eov2omoidhrgpdssgqln44jannfs5frthk41glkjd/family1/2b6de9d290d84335b95f4f90c9cf4e24 2023-02-08 08:56:21,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionFromAdmin__6t191ppsonmae4j5lvocfp0eov2omoidhrgpdssgqln44jannfs5frthk41glkjd/family1/2b6de9d290d84335b95f4f90c9cf4e24 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/950dc0ede59f434ea220df0105bc9543_SeqId_28_ 2023-02-08 08:56:21,986 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionFromAdmin__6t191ppsonmae4j5lvocfp0eov2omoidhrgpdssgqln44jannfs5frthk41glkjd/family1/2b6de9d290d84335b95f4f90c9cf4e24 into cde43f550d582d55f69773dc9ed101fa/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/950dc0ede59f434ea220df0105bc9543_SeqId_28_ - updating store file list. 2023-02-08 08:56:22,007 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/950dc0ede59f434ea220df0105bc9543_SeqId_28_ into cde43f550d582d55f69773dc9ed101fa/family1 2023-02-08 08:56:22,007 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionFromAdmin__6t191ppsonmae4j5lvocfp0eov2omoidhrgpdssgqln44jannfs5frthk41glkjd/family1/2b6de9d290d84335b95f4f90c9cf4e24 into cde43f550d582d55f69773dc9ed101fa/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/family1/950dc0ede59f434ea220df0105bc9543_SeqId_28_) 2023-02-08 08:56:22,008 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionFromAdmin__6t191ppsonmae4j5lvocfp0eov2omoidhrgpdssgqln44jannfs5frthk41glkjd/family1/2b6de9d290d84335b95f4f90c9cf4e24 2023-02-08 08:56:22,010 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230208 table='testMajorCompactionFromAdmin' column='family1' 2023-02-08 08:56:22,010 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:56:22,012 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230208560d40b9130042959a31c153457b0b6d to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230208560d40b9130042959a31c153457b0b6d 2023-02-08 08:56:22,013 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023020890db32bd69544183b5e47574b6a1d4fb to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023020890db32bd69544183b5e47574b6a1d4fb 2023-02-08 08:56:22,014 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302089f015008062147c891e3a10f7cef0118 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302089f015008062147c891e3a10f7cef0118 2023-02-08 08:56:22,015 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230208ad58748bdee74dffa67b15bb25a998ea to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230208ad58748bdee74dffa67b15bb25a998ea 2023-02-08 08:56:22,015 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-08 08:56:22,015 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230208' 2023-02-08 08:56:22,016 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230208 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-08 08:56:22,016 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230208 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-08 08:56:22,976 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230208420415326d424154b9e67382f4427de9 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230208420415326d424154b9e67382f4427de9 2023-02-08 08:56:22,977 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230208 table='testMajorCompactionFromAdmin' column='family1' 2023-02-08 08:56:22,992 INFO [MobCompactor-pool-0] client.ConnectionImplementation(1944): Getting master connection state from TTL Cache 2023-02-08 08:56:22,992 INFO [MobCompactor-pool-0] client.ConnectionImplementation(403): Getting master state using rpc call 2023-02-08 08:56:23,018 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230208/family1/8ad7489b5af249dabff932d05c6cef99 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-02-08 08:56:23,019 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885., hostname=jenkins-hbase4.apache.org,42239,1675846413348, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230208/family1/8ad7489b5af249dabff932d05c6cef99}] 2023-02-08 08:56:23,021 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(763): Validating hfile at hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230208/family1/8ad7489b5af249dabff932d05c6cef99 for inclusion in e0c4a41d57aa926b6b63e3ff74002885/family1 2023-02-08 08:56:23,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-02-08 08:56:23,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-08 08:56:23,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegion(2445): Flush status journal for e0c4a41d57aa926b6b63e3ff74002885: 2023-02-08 08:56:23,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230208/family1/8ad7489b5af249dabff932d05c6cef99 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionFromAdmin__hp8bole54tc64egpqcthdku2gkhcpbsv3l3ojeo6bli3rh1d14d9v5qlbq8imv5d/family1/8ad7489b5af249dabff932d05c6cef99 2023-02-08 08:56:23,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionFromAdmin__hp8bole54tc64egpqcthdku2gkhcpbsv3l3ojeo6bli3rh1d14d9v5qlbq8imv5d/family1/8ad7489b5af249dabff932d05c6cef99 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/15ca4b047b424fe9a6a16a80c3d450fc_SeqId_28_ 2023-02-08 08:56:23,045 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(849): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionFromAdmin__hp8bole54tc64egpqcthdku2gkhcpbsv3l3ojeo6bli3rh1d14d9v5qlbq8imv5d/family1/8ad7489b5af249dabff932d05c6cef99 into e0c4a41d57aa926b6b63e3ff74002885/family1 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/15ca4b047b424fe9a6a16a80c3d450fc_SeqId_28_ - updating store file list. 2023-02-08 08:56:23,067 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(882): Loaded HFile hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/15ca4b047b424fe9a6a16a80c3d450fc_SeqId_28_ into e0c4a41d57aa926b6b63e3ff74002885/family1 2023-02-08 08:56:23,067 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.HStore(855): Successfully loaded hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionFromAdmin__hp8bole54tc64egpqcthdku2gkhcpbsv3l3ojeo6bli3rh1d14d9v5qlbq8imv5d/family1/8ad7489b5af249dabff932d05c6cef99 into e0c4a41d57aa926b6b63e3ff74002885/family1 (new location: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/family1/15ca4b047b424fe9a6a16a80c3d450fc_SeqId_28_) 2023-02-08 08:56:23,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=42239] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/staging/jenkins__testMajorCompactionFromAdmin__hp8bole54tc64egpqcthdku2gkhcpbsv3l3ojeo6bli3rh1d14d9v5qlbq8imv5d/family1/8ad7489b5af249dabff932d05c6cef99 2023-02-08 08:56:23,069 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230208 table='testMajorCompactionFromAdmin' column='family1' 2023-02-08 08:56:23,069 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:56:23,070 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023020863aa864123db4e9e87f6815e09fbf566 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023020863aa864123db4e9e87f6815e09fbf566 2023-02-08 08:56:23,071 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302086ec9215a2b764e74b7063c9b890afff6 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302086ec9215a2b764e74b7063c9b890afff6 2023-02-08 08:56:23,072 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023020891e001a399814bdd972e719b30ea4328 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023020891e001a399814bdd972e719b30ea4328 2023-02-08 08:56:23,073 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230208c04ca662d3104207870612ea0c0f48b4 to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230208c04ca662d3104207870612ea0c0f48b4 2023-02-08 08:56:23,073 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-08 08:56:23,073 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230208' 2023-02-08 08:56:23,074 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230208 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-08 08:56:23,074 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-02-08 08:56:23,074 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-08 08:56:23,074 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302082476c309d3584d9f8af8198d02ad5eda_del] 2023-02-08 08:56:23,074 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:56:23,076 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302082476c309d3584d9f8af8198d02ad5eda_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302082476c309d3584d9f8af8198d02ad5eda_del 2023-02-08 08:56:23,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-08 08:56:23,076 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230208d478052e31ac4370a516048eb7c5f0c4_del] 2023-02-08 08:56:23,076 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:56:23,079 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230208d478052e31ac4370a516048eb7c5f0c4_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230208d478052e31ac4370a516048eb7c5f0c4_del 2023-02-08 08:56:23,079 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-08 08:56:23,079 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] compactions.PartitionedMobCompactor(367): [hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302084f4fc8360d8e428dafd1f636514bc21b_del] 2023-02-08 08:56:23,079 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] backup.HFileArchiver(360): Archiving compacted files. 2023-02-08 08:56:23,080 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302084f4fc8360d8e428dafd1f636514bc21b_del to hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c202302084f4fc8360d8e428dafd1f636514bc21b_del 2023-02-08 08:56:23,080 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-08 08:56:23,080 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1675846416473] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-08 08:56:23,082 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-08 08:56:23,083 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 3.5990 sec 2023-02-08 08:56:23,103 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 08:56:23,105 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 08:56:23,107 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-08 08:56:23,107 DEBUG [Listener at localhost/40763] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-08 08:56:23,111 INFO [Listener at localhost/40763] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-08 08:56:23,155 INFO [Listener at localhost/40763] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=247 (was 257), OpenFileDescriptor=646 (was 645) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=124 (was 130), ProcessCount=170 (was 170), AvailableMemoryMB=4235 (was 4247) 2023-02-08 08:56:23,156 DEBUG [Listener at localhost/40763] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x64b944ad to 127.0.0.1:61594 2023-02-08 08:56:23,156 DEBUG [Listener at localhost/40763] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:56:23,156 INFO [Listener at localhost/40763] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-08 08:56:23,157 INFO [Listener at localhost/40763] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-08 08:56:23,157 DEBUG [Listener at localhost/40763] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3c1e7f16 to 127.0.0.1:61594 2023-02-08 08:56:23,157 DEBUG [Listener at localhost/40763] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:56:23,157 DEBUG [Listener at localhost/40763] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-08 08:56:23,157 DEBUG [Listener at localhost/40763] util.JVMClusterUtil(257): Found active master hash=864526047, stopped=false 2023-02-08 08:56:23,157 INFO [Listener at localhost/40763] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,37869,1675846412232 2023-02-08 08:56:23,160 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-08 08:56:23,160 INFO [Listener at localhost/40763] procedure2.ProcedureExecutor(629): Stopping 2023-02-08 08:56:23,160 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-08 08:56:23,160 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 08:56:23,160 DEBUG [Listener at localhost/40763] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x29bc2cbd to 127.0.0.1:61594 2023-02-08 08:56:23,160 DEBUG [Listener at localhost/40763] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:56:23,160 INFO [Listener at localhost/40763] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,42239,1675846413348' ***** 2023-02-08 08:56:23,160 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 08:56:23,161 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-08 08:56:23,161 INFO [Listener at localhost/40763] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-08 08:56:23,161 INFO [RS:0;jenkins-hbase4:42239] regionserver.HeapMemoryManager(220): Stopping 2023-02-08 08:56:23,161 INFO [RS:0;jenkins-hbase4:42239] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-08 08:56:23,161 INFO [RS:0;jenkins-hbase4:42239] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-08 08:56:23,161 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-08 08:56:23,161 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(3304): Received CLOSE for b7daddc3345a00410a8096f6715baffd 2023-02-08 08:56:23,162 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-08 08:56:23,162 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(3304): Received CLOSE for 201d231772cef3ab7c7efad6d0fb3900 2023-02-08 08:56:23,162 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(3304): Received CLOSE for f19baf8f1950f69f65e90c2ac697eb02 2023-02-08 08:56:23,162 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b7daddc3345a00410a8096f6715baffd, disabling compactions & flushes 2023-02-08 08:56:23,162 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(3304): Received CLOSE for cde43f550d582d55f69773dc9ed101fa 2023-02-08 08:56:23,162 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:56:23,162 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(3304): Received CLOSE for 665fcb26db73b9d2ce507202c21c1ade 2023-02-08 08:56:23,163 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:56:23,163 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(3304): Received CLOSE for e0c4a41d57aa926b6b63e3ff74002885 2023-02-08 08:56:23,163 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. after waiting 0 ms 2023-02-08 08:56:23,163 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:56:23,163 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(3304): Received CLOSE for 4b057d76430e4231e80be3c284b60af5 2023-02-08 08:56:23,163 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(3304): Received CLOSE for 7e8631296641f454e22df0ec18084b7f 2023-02-08 08:56:23,163 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(3304): Received CLOSE for d6a9c249ca6bd89fbc448e7f6a1c9903 2023-02-08 08:56:23,163 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(3304): Received CLOSE for 5f0bc0fd5600fb2bfd54b18349c52a4a 2023-02-08 08:56:23,163 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(3304): Received CLOSE for b9a58ce7226f8c74217cc9288a09172f 2023-02-08 08:56:23,163 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(3304): Received CLOSE for 370d7626b49b9b3c93d59dd5a186a23e 2023-02-08 08:56:23,163 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(3304): Received CLOSE for 70773be56fb60058a14d139eb59c7fb6 2023-02-08 08:56:23,163 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(3304): Received CLOSE for 0336b616da03fa0e018dfc24ffbd4919 2023-02-08 08:56:23,163 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(3304): Received CLOSE for c4e6f28a0b52d9de77dc2ced57e02cc8 2023-02-08 08:56:23,163 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(3304): Received CLOSE for 79e77e74725bcdadb1e041ea443ff410 2023-02-08 08:56:23,163 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(3304): Received CLOSE for ddecc9a567f4f99742deec5e1f409ec6 2023-02-08 08:56:23,163 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(3304): Received CLOSE for 70af1f4bea4eb4fc084512e4e5840121 2023-02-08 08:56:23,163 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(3304): Received CLOSE for f453f93eaf555c7a1e29a37805bafe73 2023-02-08 08:56:23,163 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(3304): Received CLOSE for a6defd68663718a57a591a97ac1ac7c6 2023-02-08 08:56:23,163 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:23,163 DEBUG [RS:0;jenkins-hbase4:42239] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x43be4382 to 127.0.0.1:61594 2023-02-08 08:56:23,163 DEBUG [RS:0;jenkins-hbase4:42239] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:56:23,164 INFO [RS:0;jenkins-hbase4:42239] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-08 08:56:23,164 INFO [RS:0;jenkins-hbase4:42239] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-08 08:56:23,164 INFO [RS:0;jenkins-hbase4:42239] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-08 08:56:23,164 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-08 08:56:23,164 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-02-08 08:56:23,164 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-08 08:56:23,164 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-08 08:56:23,164 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-08 08:56:23,165 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-08 08:56:23,165 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-08 08:56:23,165 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-02-08 08:56:23,165 DEBUG [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(1505): Waiting on 0336b616da03fa0e018dfc24ffbd4919, 1588230740, 201d231772cef3ab7c7efad6d0fb3900, 370d7626b49b9b3c93d59dd5a186a23e, 4b057d76430e4231e80be3c284b60af5, 5f0bc0fd5600fb2bfd54b18349c52a4a, 665fcb26db73b9d2ce507202c21c1ade, 70773be56fb60058a14d139eb59c7fb6, 70af1f4bea4eb4fc084512e4e5840121, 79e77e74725bcdadb1e041ea443ff410, 7e8631296641f454e22df0ec18084b7f, a6defd68663718a57a591a97ac1ac7c6, b7daddc3345a00410a8096f6715baffd, b9a58ce7226f8c74217cc9288a09172f, c4e6f28a0b52d9de77dc2ced57e02cc8, cde43f550d582d55f69773dc9ed101fa, d6a9c249ca6bd89fbc448e7f6a1c9903, ddecc9a567f4f99742deec5e1f409ec6, e0c4a41d57aa926b6b63e3ff74002885, f19baf8f1950f69f65e90c2ac697eb02, f453f93eaf555c7a1e29a37805bafe73 2023-02-08 08:56:23,167 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyPolicy/b7daddc3345a00410a8096f6715baffd/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-08 08:56:23,168 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:56:23,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b7daddc3345a00410a8096f6715baffd: Waiting for close lock at 1675846583162Running coprocessor pre-close hooks at 1675846583162Disabling compacts and flushes for region at 1675846583162Disabling writes for close at 1675846583163 (+1 ms)Writing region close event to WAL at 1675846583163Running coprocessor post-close hooks at 1675846583167 (+4 ms)Closed at 1675846583168 (+1 ms) 2023-02-08 08:56:23,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1675846457942.b7daddc3345a00410a8096f6715baffd. 2023-02-08 08:56:23,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 201d231772cef3ab7c7efad6d0fb3900, disabling compactions & flushes 2023-02-08 08:56:23,168 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:56:23,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:56:23,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. after waiting 0 ms 2023-02-08 08:56:23,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:56:23,176 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/201d231772cef3ab7c7efad6d0fb3900/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-02-08 08:56:23,177 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:56:23,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 201d231772cef3ab7c7efad6d0fb3900: Waiting for close lock at 1675846583168Running coprocessor pre-close hooks at 1675846583168Disabling compacts and flushes for region at 1675846583168Disabling writes for close at 1675846583168Writing region close event to WAL at 1675846583169 (+1 ms)Running coprocessor post-close hooks at 1675846583177 (+8 ms)Closed at 1675846583177 2023-02-08 08:56:23,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1675846487343.201d231772cef3ab7c7efad6d0fb3900. 2023-02-08 08:56:23,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f19baf8f1950f69f65e90c2ac697eb02, disabling compactions & flushes 2023-02-08 08:56:23,178 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:56:23,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:56:23,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. after waiting 0 ms 2023-02-08 08:56:23,178 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:56:23,184 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/.tmp/info/a21d79b21d274cb78001156c188b4881 2023-02-08 08:56:23,187 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/f19baf8f1950f69f65e90c2ac697eb02/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-08 08:56:23,188 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:56:23,188 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f19baf8f1950f69f65e90c2ac697eb02: Waiting for close lock at 1675846583178Running coprocessor pre-close hooks at 1675846583178Disabling compacts and flushes for region at 1675846583178Disabling writes for close at 1675846583178Writing region close event to WAL at 1675846583184 (+6 ms)Running coprocessor post-close hooks at 1675846583188 (+4 ms)Closed at 1675846583188 2023-02-08 08:56:23,188 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a21d79b21d274cb78001156c188b4881 2023-02-08 08:56:23,188 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1675846535404.f19baf8f1950f69f65e90c2ac697eb02. 2023-02-08 08:56:23,188 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing cde43f550d582d55f69773dc9ed101fa, disabling compactions & flushes 2023-02-08 08:56:23,189 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:23,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:23,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. after waiting 0 ms 2023-02-08 08:56:23,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:23,192 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/cde43f550d582d55f69773dc9ed101fa/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-08 08:56:23,193 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:23,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for cde43f550d582d55f69773dc9ed101fa: Waiting for close lock at 1675846583188Running coprocessor pre-close hooks at 1675846583188Disabling compacts and flushes for region at 1675846583188Disabling writes for close at 1675846583189 (+1 ms)Writing region close event to WAL at 1675846583190 (+1 ms)Running coprocessor post-close hooks at 1675846583192 (+2 ms)Closed at 1675846583193 (+1 ms) 2023-02-08 08:56:23,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1675846565305.cde43f550d582d55f69773dc9ed101fa. 2023-02-08 08:56:23,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 665fcb26db73b9d2ce507202c21c1ade, disabling compactions & flushes 2023-02-08 08:56:23,193 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:56:23,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:56:23,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. after waiting 0 ms 2023-02-08 08:56:23,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:56:23,195 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/.tmp/rep_barrier/0fa8bbdae74c488bb25fe1ff8c6c2baa 2023-02-08 08:56:23,199 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/665fcb26db73b9d2ce507202c21c1ade/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-08 08:56:23,200 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:56:23,200 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 665fcb26db73b9d2ce507202c21c1ade: Waiting for close lock at 1675846583193Running coprocessor pre-close hooks at 1675846583193Disabling compacts and flushes for region at 1675846583193Disabling writes for close at 1675846583193Writing region close event to WAL at 1675846583194 (+1 ms)Running coprocessor post-close hooks at 1675846583200 (+6 ms)Closed at 1675846583200 2023-02-08 08:56:23,200 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1675846508433.665fcb26db73b9d2ce507202c21c1ade. 2023-02-08 08:56:23,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing e0c4a41d57aa926b6b63e3ff74002885, disabling compactions & flushes 2023-02-08 08:56:23,201 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:23,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:23,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. after waiting 0 ms 2023-02-08 08:56:23,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:23,201 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0fa8bbdae74c488bb25fe1ff8c6c2baa 2023-02-08 08:56:23,205 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/e0c4a41d57aa926b6b63e3ff74002885/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-08 08:56:23,206 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:23,206 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for e0c4a41d57aa926b6b63e3ff74002885: Waiting for close lock at 1675846583201Running coprocessor pre-close hooks at 1675846583201Disabling compacts and flushes for region at 1675846583201Disabling writes for close at 1675846583201Writing region close event to WAL at 1675846583202 (+1 ms)Running coprocessor post-close hooks at 1675846583206 (+4 ms)Closed at 1675846583206 2023-02-08 08:56:23,206 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1675846565305.e0c4a41d57aa926b6b63e3ff74002885. 2023-02-08 08:56:23,206 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 4b057d76430e4231e80be3c284b60af5, disabling compactions & flushes 2023-02-08 08:56:23,206 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:56:23,206 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:56:23,207 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. after waiting 0 ms 2023-02-08 08:56:23,207 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:56:23,209 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithWeeklyPolicy/4b057d76430e4231e80be3c284b60af5/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-08 08:56:23,211 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:56:23,211 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/.tmp/table/3b4b4766381142409589799c30bfa000 2023-02-08 08:56:23,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 4b057d76430e4231e80be3c284b60af5: Waiting for close lock at 1675846583206Running coprocessor pre-close hooks at 1675846583206Disabling compacts and flushes for region at 1675846583206Disabling writes for close at 1675846583207 (+1 ms)Writing region close event to WAL at 1675846583207Running coprocessor post-close hooks at 1675846583210 (+3 ms)Closed at 1675846583211 (+1 ms) 2023-02-08 08:56:23,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1675846442609.4b057d76430e4231e80be3c284b60af5. 2023-02-08 08:56:23,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7e8631296641f454e22df0ec18084b7f, disabling compactions & flushes 2023-02-08 08:56:23,211 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:56:23,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:56:23,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. after waiting 0 ms 2023-02-08 08:56:23,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:56:23,215 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/7e8631296641f454e22df0ec18084b7f/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-08 08:56:23,216 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:56:23,216 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7e8631296641f454e22df0ec18084b7f: Waiting for close lock at 1675846583211Running coprocessor pre-close hooks at 1675846583211Disabling compacts and flushes for region at 1675846583211Disabling writes for close at 1675846583211Writing region close event to WAL at 1675846583212 (+1 ms)Running coprocessor post-close hooks at 1675846583215 (+3 ms)Closed at 1675846583216 (+1 ms) 2023-02-08 08:56:23,216 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1675846535404.7e8631296641f454e22df0ec18084b7f. 2023-02-08 08:56:23,216 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3b4b4766381142409589799c30bfa000 2023-02-08 08:56:23,216 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing d6a9c249ca6bd89fbc448e7f6a1c9903, disabling compactions & flushes 2023-02-08 08:56:23,216 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903. 2023-02-08 08:56:23,216 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903. 2023-02-08 08:56:23,216 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903. after waiting 0 ms 2023-02-08 08:56:23,216 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903. 2023-02-08 08:56:23,216 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing d6a9c249ca6bd89fbc448e7f6a1c9903 1/1 column families, dataSize=109 B heapSize=592 B 2023-02-08 08:56:23,216 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/.tmp/info/a21d79b21d274cb78001156c188b4881 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/info/a21d79b21d274cb78001156c188b4881 2023-02-08 08:56:23,220 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-02-08 08:56:23,221 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a21d79b21d274cb78001156c188b4881 2023-02-08 08:56:23,221 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/info/a21d79b21d274cb78001156c188b4881, entries=226, sequenceid=130, filesize=37.9 K 2023-02-08 08:56:23,221 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/.tmp/rep_barrier/0fa8bbdae74c488bb25fe1ff8c6c2baa as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/rep_barrier/0fa8bbdae74c488bb25fe1ff8c6c2baa 2023-02-08 08:56:23,222 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/namespace/d6a9c249ca6bd89fbc448e7f6a1c9903/.tmp/info/26ea69d73a2b4ffc871298a52435eb23 2023-02-08 08:56:23,225 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0fa8bbdae74c488bb25fe1ff8c6c2baa 2023-02-08 08:56:23,225 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/rep_barrier/0fa8bbdae74c488bb25fe1ff8c6c2baa, entries=1, sequenceid=130, filesize=5.0 K 2023-02-08 08:56:23,226 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/.tmp/table/3b4b4766381142409589799c30bfa000 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/table/3b4b4766381142409589799c30bfa000 2023-02-08 08:56:23,226 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/namespace/d6a9c249ca6bd89fbc448e7f6a1c9903/.tmp/info/26ea69d73a2b4ffc871298a52435eb23 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/namespace/d6a9c249ca6bd89fbc448e7f6a1c9903/info/26ea69d73a2b4ffc871298a52435eb23 2023-02-08 08:56:23,229 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3b4b4766381142409589799c30bfa000 2023-02-08 08:56:23,229 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/table/3b4b4766381142409589799c30bfa000, entries=27, sequenceid=130, filesize=6.6 K 2023-02-08 08:56:23,230 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/namespace/d6a9c249ca6bd89fbc448e7f6a1c9903/info/26ea69d73a2b4ffc871298a52435eb23, entries=3, sequenceid=7, filesize=4.8 K 2023-02-08 08:56:23,230 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 65ms, sequenceid=130, compaction requested=false 2023-02-08 08:56:23,233 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for d6a9c249ca6bd89fbc448e7f6a1c9903 in 17ms, sequenceid=7, compaction requested=false 2023-02-08 08:56:23,240 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-02-08 08:56:23,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/hbase/namespace/d6a9c249ca6bd89fbc448e7f6a1c9903/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-02-08 08:56:23,240 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-08 08:56:23,241 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-08 08:56:23,241 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1675846583164Running coprocessor pre-close hooks at 1675846583164Disabling compacts and flushes for region at 1675846583164Disabling writes for close at 1675846583165 (+1 ms)Obtaining lock to block concurrent updates at 1675846583165Preparing flush snapshotting stores in 1588230740 at 1675846583165Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=51772, getHeapSize=80064, getOffHeapSize=0, getCellsCount=365 at 1675846583165Flushing stores of hbase:meta,,1.1588230740 at 1675846583165Flushing 1588230740/info: creating writer at 1675846583166 (+1 ms)Flushing 1588230740/info: appending metadata at 1675846583170 (+4 ms)Flushing 1588230740/info: closing flushed file at 1675846583170Flushing 1588230740/rep_barrier: creating writer at 1675846583188 (+18 ms)Flushing 1588230740/rep_barrier: appending metadata at 1675846583191 (+3 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1675846583191Flushing 1588230740/table: creating writer at 1675846583202 (+11 ms)Flushing 1588230740/table: appending metadata at 1675846583206 (+4 ms)Flushing 1588230740/table: closing flushed file at 1675846583206Flushing 1588230740/info: reopening flushed file at 1675846583217 (+11 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1675846583222 (+5 ms)Flushing 1588230740/table: reopening flushed file at 1675846583226 (+4 ms)Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 65ms, sequenceid=130, compaction requested=false at 1675846583230 (+4 ms)Writing region close event to WAL at 1675846583236 (+6 ms)Running coprocessor post-close hooks at 1675846583240 (+4 ms)Closed at 1675846583241 (+1 ms) 2023-02-08 08:56:23,242 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903. 2023-02-08 08:56:23,242 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for d6a9c249ca6bd89fbc448e7f6a1c9903: Waiting for close lock at 1675846583216Running coprocessor pre-close hooks at 1675846583216Disabling compacts and flushes for region at 1675846583216Disabling writes for close at 1675846583216Obtaining lock to block concurrent updates at 1675846583216Preparing flush snapshotting stores in d6a9c249ca6bd89fbc448e7f6a1c9903 at 1675846583216Finished memstore snapshotting hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903., syncing WAL and waiting on mvcc, flushsize=dataSize=109, getHeapSize=576, getOffHeapSize=0, getCellsCount=3 at 1675846583216Flushing stores of hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903. at 1675846583217 (+1 ms)Flushing d6a9c249ca6bd89fbc448e7f6a1c9903/info: creating writer at 1675846583217Flushing d6a9c249ca6bd89fbc448e7f6a1c9903/info: appending metadata at 1675846583219 (+2 ms)Flushing d6a9c249ca6bd89fbc448e7f6a1c9903/info: closing flushed file at 1675846583219Flushing d6a9c249ca6bd89fbc448e7f6a1c9903/info: reopening flushed file at 1675846583227 (+8 ms)Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for d6a9c249ca6bd89fbc448e7f6a1c9903 in 17ms, sequenceid=7, compaction requested=false at 1675846583233 (+6 ms)Writing region close event to WAL at 1675846583236 (+3 ms)Running coprocessor post-close hooks at 1675846583241 (+5 ms)Closed at 1675846583242 (+1 ms) 2023-02-08 08:56:23,242 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-08 08:56:23,242 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1675846415065.d6a9c249ca6bd89fbc448e7f6a1c9903. 2023-02-08 08:56:23,242 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 5f0bc0fd5600fb2bfd54b18349c52a4a, disabling compactions & flushes 2023-02-08 08:56:23,242 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:56:23,242 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:56:23,242 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. after waiting 0 ms 2023-02-08 08:56:23,242 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:56:23,250 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/5f0bc0fd5600fb2bfd54b18349c52a4a/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-08 08:56:23,251 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:56:23,251 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 5f0bc0fd5600fb2bfd54b18349c52a4a: Waiting for close lock at 1675846583242Running coprocessor pre-close hooks at 1675846583242Disabling compacts and flushes for region at 1675846583242Disabling writes for close at 1675846583242Writing region close event to WAL at 1675846583247 (+5 ms)Running coprocessor post-close hooks at 1675846583250 (+3 ms)Closed at 1675846583251 (+1 ms) 2023-02-08 08:56:23,251 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1675846508433.5f0bc0fd5600fb2bfd54b18349c52a4a. 2023-02-08 08:56:23,251 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b9a58ce7226f8c74217cc9288a09172f, disabling compactions & flushes 2023-02-08 08:56:23,251 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:23,251 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:23,251 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. after waiting 0 ms 2023-02-08 08:56:23,251 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:23,254 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionFromAdmin/b9a58ce7226f8c74217cc9288a09172f/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-08 08:56:23,255 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:23,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b9a58ce7226f8c74217cc9288a09172f: Waiting for close lock at 1675846583251Running coprocessor pre-close hooks at 1675846583251Disabling compacts and flushes for region at 1675846583251Disabling writes for close at 1675846583251Writing region close event to WAL at 1675846583252 (+1 ms)Running coprocessor post-close hooks at 1675846583254 (+2 ms)Closed at 1675846583255 (+1 ms) 2023-02-08 08:56:23,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1675846565305.b9a58ce7226f8c74217cc9288a09172f. 2023-02-08 08:56:23,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 370d7626b49b9b3c93d59dd5a186a23e, disabling compactions & flushes 2023-02-08 08:56:23,255 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:56:23,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:56:23,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. after waiting 0 ms 2023-02-08 08:56:23,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:56:23,258 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/370d7626b49b9b3c93d59dd5a186a23e/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-02-08 08:56:23,260 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:56:23,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 370d7626b49b9b3c93d59dd5a186a23e: Waiting for close lock at 1675846583255Running coprocessor pre-close hooks at 1675846583255Disabling compacts and flushes for region at 1675846583255Disabling writes for close at 1675846583255Writing region close event to WAL at 1675846583256 (+1 ms)Running coprocessor post-close hooks at 1675846583259 (+3 ms)Closed at 1675846583260 (+1 ms) 2023-02-08 08:56:23,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1675846470907.370d7626b49b9b3c93d59dd5a186a23e. 2023-02-08 08:56:23,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 70773be56fb60058a14d139eb59c7fb6, disabling compactions & flushes 2023-02-08 08:56:23,260 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6. 2023-02-08 08:56:23,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6. 2023-02-08 08:56:23,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6. after waiting 0 ms 2023-02-08 08:56:23,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6. 2023-02-08 08:56:23,262 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/70773be56fb60058a14d139eb59c7fb6/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-02-08 08:56:23,263 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6. 2023-02-08 08:56:23,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 70773be56fb60058a14d139eb59c7fb6: Waiting for close lock at 1675846583260Running coprocessor pre-close hooks at 1675846583260Disabling compacts and flushes for region at 1675846583260Disabling writes for close at 1675846583260Writing region close event to WAL at 1675846583260Running coprocessor post-close hooks at 1675846583263 (+3 ms)Closed at 1675846583263 2023-02-08 08:56:23,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1675846452943.70773be56fb60058a14d139eb59c7fb6. 2023-02-08 08:56:23,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 0336b616da03fa0e018dfc24ffbd4919, disabling compactions & flushes 2023-02-08 08:56:23,263 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919. 2023-02-08 08:56:23,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919. 2023-02-08 08:56:23,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919. after waiting 0 ms 2023-02-08 08:56:23,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919. 2023-02-08 08:56:23,265 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/0336b616da03fa0e018dfc24ffbd4919/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-02-08 08:56:23,266 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919. 2023-02-08 08:56:23,266 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 0336b616da03fa0e018dfc24ffbd4919: Waiting for close lock at 1675846583263Running coprocessor pre-close hooks at 1675846583263Disabling compacts and flushes for region at 1675846583263Disabling writes for close at 1675846583263Writing region close event to WAL at 1675846583263Running coprocessor post-close hooks at 1675846583266 (+3 ms)Closed at 1675846583266 2023-02-08 08:56:23,266 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1675846452943.0336b616da03fa0e018dfc24ffbd4919. 2023-02-08 08:56:23,266 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c4e6f28a0b52d9de77dc2ced57e02cc8, disabling compactions & flushes 2023-02-08 08:56:23,266 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:56:23,266 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:56:23,266 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. after waiting 0 ms 2023-02-08 08:56:23,266 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:56:23,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/ns/testMinorCompaction/c4e6f28a0b52d9de77dc2ced57e02cc8/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-08 08:56:23,269 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:56:23,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c4e6f28a0b52d9de77dc2ced57e02cc8: Waiting for close lock at 1675846583266Running coprocessor pre-close hooks at 1675846583266Disabling compacts and flushes for region at 1675846583266Disabling writes for close at 1675846583266Writing region close event to WAL at 1675846583267 (+1 ms)Running coprocessor post-close hooks at 1675846583269 (+2 ms)Closed at 1675846583269 2023-02-08 08:56:23,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1675846535404.c4e6f28a0b52d9de77dc2ced57e02cc8. 2023-02-08 08:56:23,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 79e77e74725bcdadb1e041ea443ff410, disabling compactions & flushes 2023-02-08 08:56:23,269 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:56:23,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:56:23,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. after waiting 0 ms 2023-02-08 08:56:23,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:56:23,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testCompactionWithHFileLink/79e77e74725bcdadb1e041ea443ff410/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-08 08:56:23,272 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:56:23,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 79e77e74725bcdadb1e041ea443ff410: Waiting for close lock at 1675846583269Running coprocessor pre-close hooks at 1675846583269Disabling compacts and flushes for region at 1675846583269Disabling writes for close at 1675846583269Writing region close event to WAL at 1675846583270 (+1 ms)Running coprocessor post-close hooks at 1675846583272 (+2 ms)Closed at 1675846583272 2023-02-08 08:56:23,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1675846508433.79e77e74725bcdadb1e041ea443ff410. 2023-02-08 08:56:23,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing ddecc9a567f4f99742deec5e1f409ec6, disabling compactions & flushes 2023-02-08 08:56:23,273 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6. 2023-02-08 08:56:23,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6. 2023-02-08 08:56:23,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6. after waiting 0 ms 2023-02-08 08:56:23,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6. 2023-02-08 08:56:23,275 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testScannerOnBulkLoadRefHFiles/ddecc9a567f4f99742deec5e1f409ec6/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-02-08 08:56:23,275 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6. 2023-02-08 08:56:23,275 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for ddecc9a567f4f99742deec5e1f409ec6: Waiting for close lock at 1675846583272Running coprocessor pre-close hooks at 1675846583272Disabling compacts and flushes for region at 1675846583272Disabling writes for close at 1675846583273 (+1 ms)Writing region close event to WAL at 1675846583273Running coprocessor post-close hooks at 1675846583275 (+2 ms)Closed at 1675846583275 2023-02-08 08:56:23,275 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1675846452943.ddecc9a567f4f99742deec5e1f409ec6. 2023-02-08 08:56:23,275 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 70af1f4bea4eb4fc084512e4e5840121, disabling compactions & flushes 2023-02-08 08:56:23,275 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:56:23,275 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:56:23,275 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. after waiting 0 ms 2023-02-08 08:56:23,275 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:56:23,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMinorCompactionWithMonthlyPolicy/70af1f4bea4eb4fc084512e4e5840121/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-02-08 08:56:23,278 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:56:23,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 70af1f4bea4eb4fc084512e4e5840121: Waiting for close lock at 1675846583275Running coprocessor pre-close hooks at 1675846583275Disabling compacts and flushes for region at 1675846583275Disabling writes for close at 1675846583275Writing region close event to WAL at 1675846583276 (+1 ms)Running coprocessor post-close hooks at 1675846583278 (+2 ms)Closed at 1675846583278 2023-02-08 08:56:23,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1675846553325.70af1f4bea4eb4fc084512e4e5840121. 2023-02-08 08:56:23,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f453f93eaf555c7a1e29a37805bafe73, disabling compactions & flushes 2023-02-08 08:56:23,278 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:56:23,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:56:23,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. after waiting 0 ms 2023-02-08 08:56:23,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:56:23,281 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithMonthlyPolicy/f453f93eaf555c7a1e29a37805bafe73/recovered.edits/51.seqid, newMaxSeqId=51, maxSeqId=1 2023-02-08 08:56:23,281 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:56:23,281 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f453f93eaf555c7a1e29a37805bafe73: Waiting for close lock at 1675846583278Running coprocessor pre-close hooks at 1675846583278Disabling compacts and flushes for region at 1675846583278Disabling writes for close at 1675846583278Writing region close event to WAL at 1675846583279 (+1 ms)Running coprocessor post-close hooks at 1675846583281 (+2 ms)Closed at 1675846583281 2023-02-08 08:56:23,281 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1675846432879.f453f93eaf555c7a1e29a37805bafe73. 2023-02-08 08:56:23,281 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a6defd68663718a57a591a97ac1ac7c6, disabling compactions & flushes 2023-02-08 08:56:23,281 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:56:23,281 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:56:23,281 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. after waiting 0 ms 2023-02-08 08:56:23,281 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:56:23,283 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/a6defd68663718a57a591a97ac1ac7c6/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-08 08:56:23,284 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:56:23,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a6defd68663718a57a591a97ac1ac7c6: Waiting for close lock at 1675846583281Running coprocessor pre-close hooks at 1675846583281Disabling compacts and flushes for region at 1675846583281Disabling writes for close at 1675846583281Writing region close event to WAL at 1675846583282 (+1 ms)Running coprocessor post-close hooks at 1675846583284 (+2 ms)Closed at 1675846583284 2023-02-08 08:56:23,284 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1675846416579.a6defd68663718a57a591a97ac1ac7c6. 2023-02-08 08:56:23,365 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,42239,1675846413348; all regions closed. 2023-02-08 08:56:23,373 DEBUG [RS:0;jenkins-hbase4:42239] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/oldWALs 2023-02-08 08:56:23,373 INFO [RS:0;jenkins-hbase4:42239] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42239%2C1675846413348.meta:.meta(num 1675846414830) 2023-02-08 08:56:23,377 DEBUG [RS:0;jenkins-hbase4:42239] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/oldWALs 2023-02-08 08:56:23,377 INFO [RS:0;jenkins-hbase4:42239] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42239%2C1675846413348:(num 1675846414672) 2023-02-08 08:56:23,377 DEBUG [RS:0;jenkins-hbase4:42239] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:56:23,377 INFO [RS:0;jenkins-hbase4:42239] regionserver.LeaseManager(133): Closed leases 2023-02-08 08:56:23,377 INFO [RS:0;jenkins-hbase4:42239] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-08 08:56:23,377 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-08 08:56:23,378 INFO [RS:0;jenkins-hbase4:42239] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42239 2023-02-08 08:56:23,386 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42239,1675846413348 2023-02-08 08:56:23,386 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 08:56:23,386 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-08 08:56:23,386 ERROR [Listener at localhost/40763-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@50175a51 rejected from java.util.concurrent.ThreadPoolExecutor@5428d48b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 08:56:23,386 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42239,1675846413348] 2023-02-08 08:56:23,387 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42239,1675846413348; numProcessing=1 2023-02-08 08:56:23,389 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,42239,1675846413348 already deleted, retry=false 2023-02-08 08:56:23,389 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,42239,1675846413348 expired; onlineServers=0 2023-02-08 08:56:23,389 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,37869,1675846412232' ***** 2023-02-08 08:56:23,389 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-08 08:56:23,390 DEBUG [M:0;jenkins-hbase4:37869] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@72ba94e4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-08 08:56:23,390 INFO [M:0;jenkins-hbase4:37869] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,37869,1675846412232 2023-02-08 08:56:23,390 INFO [M:0;jenkins-hbase4:37869] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,37869,1675846412232; all regions closed. 2023-02-08 08:56:23,390 DEBUG [M:0;jenkins-hbase4:37869] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-08 08:56:23,390 DEBUG [M:0;jenkins-hbase4:37869] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-08 08:56:23,390 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-08 08:56:23,390 DEBUG [M:0;jenkins-hbase4:37869] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-08 08:56:23,390 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675846414377] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1675846414377,5,FailOnTimeoutGroup] 2023-02-08 08:56:23,390 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675846414373] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1675846414373,5,FailOnTimeoutGroup] 2023-02-08 08:56:23,390 INFO [M:0;jenkins-hbase4:37869] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-08 08:56:23,390 INFO [M:0;jenkins-hbase4:37869] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-08 08:56:23,390 INFO [M:0;jenkins-hbase4:37869] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-08 08:56:23,391 DEBUG [M:0;jenkins-hbase4:37869] master.HMaster(1502): Stopping service threads 2023-02-08 08:56:23,391 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-08 08:56:23,391 INFO [M:0;jenkins-hbase4:37869] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-08 08:56:23,391 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-08 08:56:23,391 ERROR [M:0;jenkins-hbase4:37869] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-20,5,PEWorkerGroup] Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-02-08 08:56:23,392 INFO [M:0;jenkins-hbase4:37869] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-08 08:56:23,392 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-08 08:56:23,392 DEBUG [M:0;jenkins-hbase4:37869] zookeeper.ZKUtil(398): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-08 08:56:23,392 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-08 08:56:23,392 WARN [M:0;jenkins-hbase4:37869] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-08 08:56:23,392 INFO [M:0;jenkins-hbase4:37869] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-08 08:56:23,392 INFO [M:0;jenkins-hbase4:37869] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-08 08:56:23,393 DEBUG [M:0;jenkins-hbase4:37869] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-08 08:56:23,393 INFO [M:0;jenkins-hbase4:37869] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 08:56:23,393 DEBUG [M:0;jenkins-hbase4:37869] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 08:56:23,393 DEBUG [M:0;jenkins-hbase4:37869] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-08 08:56:23,393 DEBUG [M:0;jenkins-hbase4:37869] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 08:56:23,393 INFO [M:0;jenkins-hbase4:37869] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.77 KB heapSize=598.51 KB 2023-02-08 08:56:23,487 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 08:56:23,487 INFO [RS:0;jenkins-hbase4:42239] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,42239,1675846413348; zookeeper connection closed. 2023-02-08 08:56:23,488 ERROR [Listener at localhost/40763-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4a5e1ad3 rejected from java.util.concurrent.ThreadPoolExecutor@5428d48b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 08:56:23,488 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): regionserver:42239-0x1003322ce5d0001, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 08:56:23,488 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@8e3ed69] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@8e3ed69 2023-02-08 08:56:23,488 ERROR [Listener at localhost/40763-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@32ef1012 rejected from java.util.concurrent.ThreadPoolExecutor@5428d48b[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 533] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 08:56:23,488 INFO [Listener at localhost/40763] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-02-08 08:56:23,800 INFO [M:0;jenkins-hbase4:37869] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.77 KB at sequenceid=1006 (bloomFilter=true), to=hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/5de218e2d9ed4d3a9f9a5ff2caf7cb43 2023-02-08 08:56:23,804 INFO [M:0;jenkins-hbase4:37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5de218e2d9ed4d3a9f9a5ff2caf7cb43 2023-02-08 08:56:23,804 DEBUG [M:0;jenkins-hbase4:37869] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/5de218e2d9ed4d3a9f9a5ff2caf7cb43 as hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/5de218e2d9ed4d3a9f9a5ff2caf7cb43 2023-02-08 08:56:23,808 INFO [M:0;jenkins-hbase4:37869] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5de218e2d9ed4d3a9f9a5ff2caf7cb43 2023-02-08 08:56:23,808 INFO [M:0;jenkins-hbase4:37869] regionserver.HStore(1080): Added hdfs://localhost:33187/user/jenkins/test-data/23ffb98f-79f9-98d2-9746-ec843b950004/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/5de218e2d9ed4d3a9f9a5ff2caf7cb43, entries=123, sequenceid=1006, filesize=33.3 K 2023-02-08 08:56:23,809 INFO [M:0;jenkins-hbase4:37869] regionserver.HRegion(2947): Finished flush of dataSize ~512.77 KB/525076, heapSize ~598.49 KB/612856, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 416ms, sequenceid=1006, compaction requested=false 2023-02-08 08:56:23,810 INFO [M:0;jenkins-hbase4:37869] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-08 08:56:23,810 DEBUG [M:0;jenkins-hbase4:37869] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1675846583393Disabling compacts and flushes for region at 1675846583393Disabling writes for close at 1675846583393Obtaining lock to block concurrent updates at 1675846583393Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1675846583393Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=525076, getHeapSize=612856, getOffHeapSize=0, getCellsCount=1161 at 1675846583393Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1675846583393Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1675846583393Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1675846583396 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1675846583396Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1675846583805 (+409 ms)Finished flush of dataSize ~512.77 KB/525076, heapSize ~598.49 KB/612856, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 416ms, sequenceid=1006, compaction requested=false at 1675846583809 (+4 ms)Writing region close event to WAL at 1675846583810 (+1 ms)Closed at 1675846583810 2023-02-08 08:56:23,812 INFO [M:0;jenkins-hbase4:37869] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-08 08:56:23,812 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-08 08:56:23,813 INFO [M:0;jenkins-hbase4:37869] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37869 2023-02-08 08:56:23,815 DEBUG [M:0;jenkins-hbase4:37869] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,37869,1675846412232 already deleted, retry=false 2023-02-08 08:56:23,917 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 08:56:23,917 INFO [M:0;jenkins-hbase4:37869] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,37869,1675846412232; zookeeper connection closed. 2023-02-08 08:56:23,917 ERROR [Listener at localhost/40763-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4004650d rejected from java.util.concurrent.ThreadPoolExecutor@1a83ec43[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 08:56:23,917 DEBUG [Listener at localhost/40763-EventThread] zookeeper.ZKWatcher(600): master:37869-0x1003322ce5d0000, quorum=127.0.0.1:61594, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-08 08:56:23,917 ERROR [Listener at localhost/40763-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@672cf804 rejected from java.util.concurrent.ThreadPoolExecutor@1a83ec43[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 907] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-08 08:56:23,919 WARN [Listener at localhost/40763] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-08 08:56:23,921 INFO [Listener at localhost/40763] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-08 08:56:24,027 WARN [BP-196993714-172.31.14.131-1675846409146 heartbeating to localhost/127.0.0.1:33187] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-08 08:56:24,027 WARN [BP-196993714-172.31.14.131-1675846409146 heartbeating to localhost/127.0.0.1:33187] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-196993714-172.31.14.131-1675846409146 (Datanode Uuid 0fbfb9d4-86e7-4262-8b59-c196f8c06b5d) service to localhost/127.0.0.1:33187 2023-02-08 08:56:24,030 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/cluster_d5539413-62cb-b490-c3d3-1723c9563de1/dfs/data/data1/current/BP-196993714-172.31.14.131-1675846409146] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 08:56:24,030 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/15003daa-ade9-d3ff-a914-963c41d1c2ca/cluster_d5539413-62cb-b490-c3d3-1723c9563de1/dfs/data/data2/current/BP-196993714-172.31.14.131-1675846409146] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-08 08:56:24,056 INFO [Listener at localhost/40763] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-08 08:56:24,187 INFO [Listener at localhost/40763] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-08 08:56:24,275 INFO [Listener at localhost/40763] hbase.HBaseTestingUtility(1293): Minicluster is down